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Why  GAO  Did  This  Study 

DOD  requested  $168  billion  in  fiscal 
year  2014  to  develop,  test,  and  acquire 
weapon  systems  and  other  products 
and  equipment.  About  40  percent  of 
that  total  is  for  major  defense 
acquisition  programs  or  ACAT  I 
programs.  DOD  also  invests  in  other, 
non-major  ACAT  II  and  III  programs 
that  are  generally  less  costly  at  the 
individual  program  level.  These 
programs  typically  have  fewer 
reporting  requirements  and  are 
overseen  at  lower  organizational  levels 
than  ACAT  I  programs,  although  they 
may  have  annual  funding  needs  that 
are  just  as  significant. 

GAO  was  asked  to  examine  ACAT  II 
and  III  programs.  This  report 
addresses,  among  other  issues,  (1)  the 
extent  to  which  information  is  available 
on  the  number,  cost,  and  performance 
of  ACAT  II  and  III  programs  and  (2) 
factors  that  affected  the  performance 
of  selected  ACAT  II  and  III  programs. 
GAO  collected  program  and  cost  data 
on  current  ACAT  II  and  III  programs 
from  five  DOD  components.  GAO  also 
selected  a  non-generalizable  sample  of 
15  programs  based  on  program  cost 
and  other  criteria  and  reviewed 
documentation  and  interviewed 
officials  about  program  performance. 

What  GAO  Recommends 

GAO  recommends  that  DOD  establish 
guidelines  on  what  constitutes  a 
current  ACAT  II  and  III  program,  take 
steps  to  improve  data  reliability,  and 
determine  how  to  measure  cost  and 
schedule  performance.  DOD  partially 
concurred  with  the  recommendations 
and  described  actions  it  plans  to  take. 
However,  as  discussed  in  the  report, 
DOD’s  planned  actions  may  not  fully 
address  the  issues  that  GAO  identified. 

View  GAO-15-188.  For  more  information, 
contact  Mike  Sullivan  at  202-512-4841  or 
sullivanm@gao.gov. 


What  GAO  Found 

The  Department  of  Defense  (DOD)  could  not  provide  sufficiently  reliable  data  for 
GAO  to  determine  the  number,  total  cost,  or  performance  of  DOD’s  current 
acquisition  category  (ACAT)  II  and  III  programs.  These  non-major  programs 
range  from  a  multibillion  dollar  aircraft  radar  modernization  program  to  soldier 
clothing  and  protective  equipment  programs  in  the  tens  of  millions  of  dollars. 

GAO  found  that  the  accuracy,  completeness,  and  consistency  of  DOD's  data  on 
these  programs  were  undermined  by  widespread  data  entry  issues,  missing  data, 
and  inconsistent  identification  of  current  ACAT  II  and  III  programs.  See  the  figure 
below  for  selected  data  reliability  issues  GAO  identified. 


Two  Most  Frequent  Reliability  Issues  Identified  by  GAO  in  DOD-Reported  Data  for  Acquisition 
Category  (ACAT)  II  and  III  Programs 


DOD  component-reported 
ACAT  II  and  III  programs 


Reported  programs  identified  Reported  programs  identified 
as  having  missing  data  as  not  a  current 

ACAT  II  or  III  program 


Source:  GAO  analysis  of  DOD  component  data.  |  GAO-15-188 

DOD  components  are  taking  steps  to  improve  ACAT  II  and  III  data,  but  these 
steps  do  not  fully  address  the  problems  GAO  identified.  For  example,  the 
components  have  not  established  systematic  processes  to  perform  data  quality 
tests  and  assess  the  results  to  help  identify  problems  for  further  review.  These 
types  of  tests  and  assessments  can  be  an  important  step  in  determining  whether 
data  can  be  used  for  its  intended  purposes.  Additionally,  DOD  lacks  metrics  to 
assess  ACAT  II  and  III  cost  and  schedule  performance  trends  across  programs 
and  in  some  cases  was  missing  baseline  cost  and  schedule  data  to  measure 
performance.  Having  timely  and  reliable  cost,  schedule,  and  performance  data 
on  smaller  acquisition  programs  is  critical  to  ensuring  that  DOD  and  its 
components  can  account  for  how  they  are  spending  their  money  and  how  well 
they  are  spending  it.  Reliable  data  are  also  essential  for  effective  oversight  and 
bringing  the  right  oversight  resources  to  bear  when  programs  approach  the  cost 
threshold  to  become  a  major  defense  acquisition  program  due  to  cost  growth. 

Thirteen  of  the  15  ACAT  II  or  III  programs  GAO  reviewed  in-depth  had  exceeded 
their  original  cost  or  schedule  targets.  Program  officials  from  ACAT  II  and  III 
programs  GAO  reviewed  cited  changing  performance  requirements,  testing 
issues,  quantity  changes,  and  flaws  in  original  cost  estimates,  among  other 
factors,  as  the  reasons  for  cost  and  schedule  growth.  GAO  has  previously  found 
that  similar  factors  affect  the  performance  of  major  acquisition  programs. 
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GAO 

441  G  St.  N.W. 
Washington,  DC  20548 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


March  2,  2015 

The  Honorable  Mac  Thornberry 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

Department  of  Defense  (DOD)  weapon  system  acquisition  represents  one 
of  the  largest  areas  of  the  federal  government’s  discretionary  spending.  In 
fiscal  year  2014,  DOD  requested  $168  billion  to  develop,  test,  and  acquire 
weapon  systems  and  other  products  and  equipment.  About  40  percent  of 
that  total  was  for  major  defense  acquisition  programs  (MDAP)  or 
acquisition  category  (ACAT)  I  programs.1  The  remaining  approximately 
60  percent  of  the  budget  request  included,  among  other  investments, 
funding  for  DOD’s  non-major  ACAT  II  and  III  programs.2  These  programs, 
which  include  everything  from  a  multibillion  dollar  aircraft  radar 
modernization  program  to  soldier  clothing  and  protective  equipment 
programs  in  the  tens  of  millions  of  dollars,  are  generally  less  costly  than 
MDAPs  at  the  individual  program  level.  Due  to  the  lower  level  of 
investment  involved  on  a  program-by-program  basis,  ACAT  II  and  III 
programs  typically  have  fewer  reporting  and  documentation  requirements 
and  are  overseen  at  lower  organizational  levels  than  MDAP  and  major 
automated  information  system  (MAIS)  programs.3  Accordingly, 

Congress’s  and  DOD’s  insight  into  the  performance  of  these  programs  is 
more  limited. 


^CAT  I  programs  have  an  estimated  total  acquisition  cost  of  more  than  $480  million  for 
research,  development,  test,  and  evaluation  (RDT&E)  or  more  than  $2.79  billion  for 
procurement  in  fiscal  year  2014  constant  dollars. 

2ACAT  II  programs  have  an  estimated  total  cost  of  more  than  $185  million  for  RDT&E  or 
more  than  $835  million  for  procurement  in  fiscal  year  2014  constant  dollars.  ACAT  III 
programs  are  all  other  programs  that  do  not  meet  the  criteria  for  ACAT  II  or  above.  DOD 
can  also  designate  programs  to  higher  ACAT  levels  based  on  special  interest. 

3DOD  IT  investments  that  fall  within  one  of  the  following  categories  are  designated  as 
MAIS  programs:  (1)  program  costs  in  any  single  fiscal  year  exceed  $40  million,  (2)  total 
program  acquisition  costs  exceed  $165  million,  or  (3)  total  life-cycle  costs  exceed  $520 
million  in  fiscal  year  2014  constant  dollars.  MAIS  programs  are  also  referred  to  as  ACAT 
IA  programs. 
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You  asked  us  to  examine  the  number,  acquisition  cost,  and  performance 
of  DOD’s  ACAT  II  and  III  programs.  This  report  assesses  (1)  the  extent  to 
which  information  is  available  on  the  number  of  current  ACAT  II  and  III 
programs,  their  total  estimated  acquisition  cost,  and  their  cost  and 
schedule  performance;  (2)  the  factors  affecting  the  cost  and  schedule 
performance  of  selected  ACAT  II  and  III  programs;  and  (3)  the  number  of 
DOD’s  current  ACAT  II  and  III  programs  that  are  likely  to  become 
MDAPs. 

To  determine  the  extent  to  which  information  was  available  on  the 
number  of  current  ACAT  II  and  III  programs  and  their  estimated 
acquisition  costs,  we  used  a  data  collection  instrument  to  collect  data  on 
the  number  and  cost  of  current  ACAT  II  and  III  programs  from  five  DOD 
components:  Army,  Air  Force,  Navy,  U.S.  Special  Operations  Command 
(SOCOM),  and  the  DOD  Chemical  and  Biological  Defense  Program 
(CBDP).4  Our  observations  on  DOD’s  ACAT  II  and  III  program  data  are 
based  on  the  original  data  submitted  by  the  components.5  To  assess 
information  available  on  cost  and  schedule  performance,  we  collected 
and  analyzed  acquisition  program  baseline  (APB)  documents  for  a  non- 
generalizable  random  sample  of  170  non-automated  information  system 
ACAT  II  and  III  programs.6  To  assess  the  reliability  of  the  data,  we 
reviewed  the  data  for  missing  values  and  obvious  errors,  and  compared 
the  cost  data  for  our  sample  of  170  programs  to  source  documents  when 
available.  We  found  the  data  were  unreliable  as  further  discussed  in  the 
report. 


4These  five  components  accounted  for  approximately  88  percent  of  DOD’s  requested 
RDT&E  and  procurement  funding  in  the  President’s  Fiscal  Year  2014  budget  request. 
CBDP  is  a  special  interest  program  under  the  Defense  Acquisition  Executive’s  oversight. 
The  Secretary  of  the  Army  is  the  DOD  Executive  Agent  for  CBDP.  For  the  purposes  of  this 
report,  we  refer  to  CBDP  as  a  component.  Navy  guidance  allows  for  programs  to  be 
categorized  as  ACAT  IV  programs.  For  the  purposes  of  this  report,  the  term  “ACAT  III 
program”  also  includes  Navy  and  Marine  Corps  ACAT  IV  acquisition  programs. 

5From  January  to  July  2014,  we  worked  with  components  to  attempt  to  correct  problems 
we  identified  in  the  data.  However,  we  continued  to  identify  additional  errors  and  therefore 
determined  we  would  not  be  able  to  report  the  corrected  data.  We  analyzed  the  original 
data  because  it  reflects  the  information  DOD  would  have  had  available  on  ACAT  II  and  III 
programs  at  the  time  we  collected  data. 

6While  our  intention  was  to  select  a  generalizable  sample,  after  selecting  our  sample  we 
determined  through  our  data  reliability  assessment  that  the  population  of  current  ACAT  II 
and  III  programs  could  not  be  reliably  determined. 
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To  identify  the  factors  that  affected  the  cost  and  schedule  performance  of 
selected  ACAT  II  and  III  programs,  we  analyzed  factors  cited  in  program 
documentation  and  by  DOD  program  officials.  We  reviewed  a  non- 
generalizable  sample  of  15  programs  including  each  component’s  largest 
ACAT  II  and  III  program  based  on  data  reported  by  DOD  components  and 
one  additional  program  per  component  based  on  factors  such  as  cost 
growth  or  being  part  of  a  family  of  related  systems.7  For  each  program, 
we  analyzed  APBs,  reviewed  program  documents,  and  interviewed 
program  officials  to  assess  and  identify  factors  that  affected  cost  or 
schedule  performance. 

To  determine  the  number  of  current  ACAT  II  and  III  programs  that  were 
likely  to  become  MDAPs,  we  analyzed  data  provided  by  DOD 
components  through  our  data  collection  instrument  to  identify  programs 
that  appeared  to  be  within  10  percent  of  or  to  have  exceeded  the  ACAT  I 
threshold  for  research,  development,  test,  and  evaluation  (RDT&E)  or 
procurement.  We  then  collected  information  about  these  programs  from 
DOD  components  using  a  set  of  structured  questions.  We  determined  the 
data  were  sufficiently  reliable  to  serve  as  a  starting  point  to  identify  the 
minimum  number  of  programs  likely  to  become  MDAPs  because  we  were 
able  to  confirm  data  with  relevant  program  offices.  Appendix  I  provides 
additional  details  on  our  scope  and  methodology. 

We  conducted  this  performance  audit  from  October  2013  to  March  2015 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 


Background 


DOD  acquisition  policy  defines  an  acquisition  program  as  a  directed, 
funded  effort  that  provides  a  new,  improved,  or  continuing  materiel, 
weapon,  or  information  system,  or  a  service  capability  in  response  to  an 
approved  need.8  As  shown  in  table  1,  defense  acquisition  programs  are 


7We  excluded  automated  information  systems  at  all  components  and  vaccine  programs  at 
CBDP  from  our  case  studies  because  our  focus  was  on  the  performance  of  weapon 
system  programs. 

8DOD  Directive  5000.01,  The  Defense  Acquisition  System  (Nov.  20,  2007). 
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classified  into  acquisition  categories  that  depend  on  the  value  and  type  of 
acquisition.  The  Army,  Navy,  Air  Force,  and  SOCOM  also  have 
supplemental  acquisition  policies  that  address  certain  aspects  of 
acquisition  program  categorization  and  management. 


Table  1:  Description  of  Acquisition  Category  (ACAT)  I  -  III  Programs 


ACAT 

Reason  for  ACAT  designation 

ACAT  1 

• 

• 

Major  defense  acquisition  program  estimated  to  require  an  eventual  total  expenditure  for  research,  development, 
test,  and  evaluation  (RDT&E)  of  more  than  $480  million  or,  for  procurement  of  more  than  $2.79  billion  (in  fiscal 
year  2014  dollars)  for  all  increments 

Milestone  decision  authority  designation 

ACAT  IA 

• 

• 

Major  automated  information  system  (AIS)  that  is  estimated  to  exceed: 

•  $40  million  (in  fiscal  year  2014  dollars)  for  all  expenditures  directly  related  to  the  AIS  definition,  design, 
development,  and  deployment  and  incurred  in  a  single  fiscal  year;  or 

•  $165  million  (in  fiscal  year  2014  dollars)  for  all  expenditures  directly  related  to  the  AIS  definition,  design, 
development,  and  deployment  and  incurred  from  the  beginning  of  the  materiel  solution  analysis  phase 
through  deployment  at  all  sites;  or 

•  $520  million  (in  fiscal  year  2014  dollars)  for  all  expenditures  directly  related  to  AIS  definition,  design, 
development,  deployment,  operations  and  maintenance,  and  incurred  from  the  beginning  of  the  materiel 
solution  analysis  phase  through  sustainment  for  the  estimated  useful  life  of  the  system 

Milestone  decision  authority  designation 

ACAT  II 

• 

• 

• 

Does  not  meet  criteria  for  ACAT  1  or  IA 

Major  system  estimated  to  require  an  eventual  total  expenditure  for  RTD&E  of  more  than  $1 85  million,  or  for 
procurement  of  more  than  $835  million  (in  fiscal  year  2014  dollars) 

Milestone  decision  authority  designation 

ACAT  III 

• 

• 

Does  not  meet  criteria  for  ACAT  II  or  above 

An  AIS  program  that  is  not  a  major  AIS  program 

Source:  Interim  Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System,  (Nov.  25,  2013).  |  GAO-15-188 

Notes:  The  Milestone  Decision  Authority  is  the  designated  individual  with  overall  responsibility  for  a 
program  and  approves  entry  of  a  program  into  the  next  phase  of  the  acquisition  process.  All  dollars 
figures  above  reflect  fiscal  year  2014  constant  dollars. 


ACAT  II  and  III  programs  encompass  a  wide  range  of  efforts  and  program 
sizes.  Programs  may  range  from  an  ACAT  II  program  with  a  total 
acquisition  cost  of  more  than  $3  billion  to  an  ACAT  III  program  with  an 
acquisition  cost  in  the  millions  of  dollars  or  lower.  DOD’s  acquisition  policy 
does  not  establish  a  minimum  cost  for  ACAT  III  programs. 

The  level  of  oversight  for  acquisition  programs  varies  based  on  the 
assigned  ACAT  level.  DOD  and  component  acquisition  policies  specify 
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the  organizational  level  of  the  milestone  decision  authority — the 
designated  individual  with  overall  responsibility  for  a  program — for  each 
ACAT  level.9  The  organizational  level  at  which  program  requirements  and 
requirements  changes  are  approved  may  vary  by  ACAT  level  as  well.  The 
organizational  level  of  the  milestone  decision  authority  for  Air  Force  ACAT 
l-lll  programs  is  shown  in  figure  1  as  an  example. 


Figure  1 :  Acquisition  Milestone  Decision  Authorities  for  Air  Force  Acquisition 
Category  (ACAT)  I  -  III  Programs 


Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics 

Defense  Acquisition  Executive 


ACAT  I 


Assistant  Secretary  of  the  Air  Force 
(Acquisition) 

Service  or  Component  Acquisition  Executive 


ACAT  II 

ACAT  I,  if  delegated 


Program  Executive  Officer 

A  military  or  civilian  official  who  has 
responsibility  for  a  specific  program 
or  portfolio  of  similar  programs 


Program  Executive  Officer 

A  military  or  civilian  official  who  has 
responsibility  for  a  specific  program 
or  portfolio  of  similar  programs 


ACAT  III 

ACAT  II,  if  delegated 


Source:  Air  Force  Instruction  63-101/20-101,  Integrated  Life  Cycle  Management  (Mar.  7,  2013).  |  GAO-15-188 


According  to  DOD  Directive  5000.01,  the  milestone  decision  authority  shall  have  the 
authority  to  approve  entry  of  an  acquisition  program  into  the  next  phase  of  the  acquisition 
process  and  shall  be  accountable  for  cost,  schedule,  and  performance  reporting  to  higher 
authority,  including  congressional  reporting. 
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All  acquisition  programs  are  required  by  statute  or  DOD  guidance  to 
provide  program  information  at  milestones  and  other  decision  points, 
although  these  requirements  differ  by  ACAT  level.  MDAP  and  MAIS 
programs,  also  known  as  ACAT  I  and  IA  programs,  require  more 
documentation  and  analysis  to  support  program  decisions  and  have  to 
regularly  report  to  Congress  on  their  cost,  schedule,  and  technical 
performance.  These  programs  are  required  to  enter  and  maintain 
program  cost,  schedule,  and  performance  data  and  create  APBs  within 
DOD’s  Defense  Acquisition  Management  Information  Retrieval  (DAMIR) 
system,  a  web-based  data  system  intended  to  provide  data  transparency 
of  acquisition  management  information  across  DOD.  Components  may 
use  DAMIR  for  other  programs,  but  it  is  not  required.  Appendix  II  provides 
additional  detail  on  acquisition  documentation  requirements  and 
congressional  reporting  requirements  by  ACAT  level. 


DOD  Does  Not  Have 
Reliable  Data  on 
ACAT  II  and  III 
Programs  and 
Component  Actions  to 
Improve  Data  Do  Not 
Fully  Address 
Limitations 


DOD  components  could  not  provide  sufficiently  reliable  data  for  us  to 
accurately  determine  the  number,  total  cost,  or  performance  of  DOD’s 
current  ACAT  II  and  III  programs.  We  found  that  the  accuracy, 
completeness,  and  consistency  of  DOD’s  data  was  undermined  by  (1) 
widespread  data  entry  issues  and  missing  data,  and  (2)  inconsistent 
identification  of  current  ACAT  II  and  III  programs  across  and  within 
components.  DOD  components  have  taken  some  steps  to  improve  ACAT 
II  and  III  program  data,  but  their  efforts  do  not  fully  address  the  causes  of 
the  problems  we  identified.  In  addition  to  data  reliability  problems,  DOD 
lacks  consistent  cost  and  schedule  metrics  across  components  to  assess 
ACAT  II  and  III  program  performance.  Further,  the  lack  of  baseline  cost 
and  schedule  data  and  comparable  schedule  milestones  prevents  DOD 
from  consistently  measuring  the  performance  of  ACAT  II  and  III 
programs.  Taken  together,  these  issues  limit  the  utility  of  DOD’s  data  on 
ACAT  II  and  III  programs  for  oversight,  decision-making,  and  reporting 
purposes. 
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DOD  Components  Cannot 
Provide  Accurate, 
Complete,  or  Consistent 
Data  on  Current  ACAT  II 
and  III  Programs 


Widespread  Data  Entry  Issues 
and  Missing  Data 


We  identified  data  reliability  issues  related  to  accuracy,  completeness,  or 
consistency  with  data  for  about  60  percent  of  ACAT  II  and  III  programs 
reported  to  us  by  DOD  components.i * * * * * * * * 10  These  issues  prevented  us  from 
accurately  determining  the  number,  total  cost,  or  performance  of  DOD’s 
current  ACAT  II  and  III  programs.  According  to  DOD  acquisition  policy, 
complete  and  current  program  information  is  essential  to  the  acquisition 
process.11  Internal  control  standards  for  federal  executive  branch 
agencies  also  emphasize  that  agencies  should  have  relevant,  reliable, 
and  timely  information  for  decision-making  and  external  reporting 
purposes.12  We  found  obvious  accuracy  and  completeness  issues  in 
program  cost  data  for  ACAT  II  and  III  programs  reported  to  us  by  DOD 
components.  We  also  observed  consistency  issues  in  program  data 
across  components  and  within  some  components  that  affected  the 
comparability  of  the  data. 

Inaccurate  data  were  evident  across  all  of  the  components;  issues  we 
observed  included  reported  dollar  values  outside  the  range  of  ACAT  II 
and  III  programs  and  basic  math  errors.  Inaccuracies  like  these  suggest 
overall  data  quality  problems.  Further,  when  we  reviewed  a  sample  of 
programs  and  compared  reported  cost  estimates  to  source  documents, 
we  found  that  cost  estimate  data  was  often  misreported.  Components 
incorrectly  reported  data  or  data  was  missing  for  64  out  of  95  programs 
for  which  we  had  complete  source  documents  in  our  non-generalizable 
sample.13  We  also  observed  missing  data  elements  to  varying  degrees  at 
all  of  the  components  except  CBDP.  For  example,  333  out  of  836 
programs  reported  by  the  components  were  missing  one  or  more  cost 
estimate  elements  or  basic  information  such  as  the  ACAT  level.  Lastly,  in 
numerous  instances  components  did  not  follow  the  instructions  of  the 


i  n 

Accuracy,  completeness,  and  consistency  are  key  characteristics  of  reliable  data  and 

refer  to  (1)  the  extent  that  recorded  data  reflect  the  actual  underlying  information;  (2)  data 

elements  for  each  program  are  populated  appropriately;  and  (3)  the  need  to  obtain  and 

use  data  that  are  clear  and  well  defined  enough  to  yield  similar  results  in  similar  analyses, 

respectively. 

^Department  of  Defense  Directive  5000.01,  The  Defense  Acquisition  System,  end.  1, 

para  E1.1 .20  at  8  (May  12,  2003)  (Certified  Current  as  of  Nov.  20,  2007). 

12GAO,  Standards  for  Internal  Control  in  the  Federal  Government,  GAO/AIMD-OO-21.3.1 

(Washington,  D.C.:  November  1999). 

13The  non-generalizable  sample  included  170  programs.  See  appendixes  I  and  III, 

respectively,  for  further  details  about  our  scope  and  methodology  and  data  reliability 

assessment. 
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data  collection  instrument,  which  also  affected  our  ability  to  use  the  data. 
See  table  2  for  examples  of  the  reliability  issues  we  identified. 

Table  2:  Examples  of  Data  Reliability  Issues  Identified  in  Component-Reported  Acquisition  Category  (ACAT)  II  and  III 

Programs 

Data  reliability  issue 

Example 

Dollar  values  outside  the  range  for  ACAT  II 
and  III  programs 

Thirty  Air  Force  programs  reported  $0  for  their  total  baseline  or  current  cost  estimates 
even  though  Department  of  Defense  policy  defines  acquisition  programs  as  funded 
efforts.  Most  of  these  programs  were  either  not  current  programs  or  were  low  cost 
efforts  to  modify  aircraft.  Air  Force  officials  confirmed  that  programs  without  funding 
should  not  be  listed  as  ACAT  programs  and  told  us  that  they  are  working  to  remove 
these  types  of  programs  from  the  component  data  system. 

Basic  math  errors 

Even  after  accounting  for  potential  rounding  errors,  there  were  54  programs  with  current 
or  baseline  total  estimates  that  did  not  match  the  sum  of  the  estimate  elements  (i.e.,  the 
total  estimate  provided  did  not  match  the  sum  of  the  cost  estimates  for  research, 
development,  test,  and  evaluation,  procurement,  acquisition  operation  and 
maintenance,  and  military  construction). 

Misreported  cost  data 

Fifty  out  of  95  programs  for  which  we  had  complete  source  documents  in  our  non- 
generalizable  sample  misreported  at  least  one  element  of  the  program’s  baseline  or 
current  cost  estimate. 

Missing  key  data  elements 

One  Army  Program  Executive  Office  initially  did  not  report  any  current  cost  estimates  for 
its  28  reported  ACAT  II  and  III  programs.  As  a  result,  we  could  not  conduct  cost 
performance  analysis  for  any  of  these  programs. 

Basic  errors  in  following  data  entry 
instructions 

Sixteen  U.S.  Special  Operations  Command  (SOCOM)  programs  reported  current  cost 
estimates  that  were  substantially  lower  than  their  baseline  cost  estimate.  For  example, 
SOCOM  reported  a  current  cost  estimate  of  $1 1 .5  million  for  its  Special 

Communications  Enterprise  program,  more  than  a  90  percent  decrease  from  the 
baseline  cost  estimate,  but  later  provided  a  corrected  estimate  of  $239  million.  In  some 
cases,  these  estimates  appeared  to  reflect  the  amount  requested  in  fiscal  year  2014 
rather  than  the  program’s  total  cost  estimate  per  the  data  collection  instrument 
instructions. 

Source:  GAO  analysis  of  Department  of  Defense  component-reported  ACAT  II  and  III  program  data.  |  GAO-15-188 


In  some  instances,  the  accuracy  and  completeness  issues  we  identified 
were  consistent  with  known  limitations  of  information  in  component 
systems.  For  example,  Army  acquisition  officials  told  us  that  in  the  past 
there  has  been  conflicting  guidance  about  whether  completed  programs 
should  be  deleted  from  the  Army’s  acquisition  information  system,  and 
some  completed  programs  were  never  removed  from  the  system. 
Officials  at  all  components,  except  CBDP,  further  told  us  that  accuracy  of 
data  in  their  systems  relies  primarily  on  the  quality  of  information 
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submitted  by  the  Program  Executive  Officer  (PEO)  or  program  offices.14 
Army,  Navy,  and  Air  Force  acquisition  officials  told  us  that  they  work  with 
PEOs  to  address  data  quality  problems,  such  as  by  conducting  ad  hoc 
checks  to  flag  obvious  errors  or  missing  data  elements  and  following  up 
with  PEOs  as  necessary.  SOCOM  acquisition  officials  told  us  the 
Acquisition  Executive  emphasizes  the  importance  of  maintaining  up-to- 
date  data  to  program  managers.  Based  on  the  data  provided  to  us  by 
DOD  components,  these  existing  data  quality  practices  are  insufficient  to 
ensure  the  accuracy  and  completeness  of  data  on  DOD’s  ACAT  II  and  III 
programs  as  required  by  DOD  policy  and  federal  internal  control 
standards. 


Inconsistent  Identification  of 
Current  ACAT  II  and  III 
Programs  Across  and  Within 
Components 


Data  provided  in  response  to  our  request  for  information  on  current  ACAT 
II  and  III  programs  (1)  included  acquisitions  that  were  not  “current”  ACAT 
II  and  III  programs  in  accordance  with  our  definition,  (2)  likely  reflected 
inconsistent  reporting  of  acquisitions  at  lower  dollar  levels  across 
components,  and  (3)  likely  excluded  certain  acquisitions  considered 
current  ACAT  II  and  III  programs  in  accordance  with  certain  component 
acquisition  policies.  For  the  purposes  of  our  report,  we  define  a  current 
program  as  one  that  has  been  formally  initiated  in  the  acquisition  process, 
but  has  not  yet  delivered  90  percent  of  its  items  or  made  90  percent  of 
planned  expenditures.  This  definition  is  consistent  with  statutory  reporting 
thresholds  used  by  Congress  in  its  reporting  requirements  for  current 
MDAP  programs.15 


Inconsistent  interpretations  of  what  constitutes  a  current  program  and  the 
inability  of  some  components  to  reliably  identify  these  programs 
contributed  to  the  inclusion  of  programs  that  were  not  current  in  the  data 
that  components  reported  to  us.  DOD  acquisition  policy  and  component 
guidance  generally  do  not  define  which  ACAT  II  or  III  programs  are 
considered  to  be  current  for  management  and  reporting  purposes.  For 
example,  DOD  acquisition  policy  defines  when  a  program  is  formally 
initiated  and  its  operations  and  support  phase  begins,  but  it  does  not 
identify  when  a  program  should  be  considered  current.  Further,  some 


14AII  CBDP  ACAT  II  and  III  acquisitions  are  managed  by  a  single  PEO,  the  Joint  Program 
Executive  Office  for  Chemical  and  Biological  Defense.  CBDP  does  not  have  a  data 
system  that  includes  ACAT  II  and  III  programs’  acquisition  baseline  cost  estimates. 
According  to  CBDP  officials,  in  response  to  our  request,  ACAT  II  and  III  program  data 
were  manually  entered  into  the  data  collection  instrument  from  hard  copy  documents. 

1510  U.S.C.  §§  2432(g)  and  (h)(1). 
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components  told  us  PEOs  may  have  different  interpretations  of  what 
constitutes  a  current  program.  Components  also  told  us  that  they  could 
not  consistently  use  the  information  in  their  data  systems  to  readily 
identify  current  programs.  Of  the  836  programs  initially  reported  by  the 
five  components,  we  identified  199  programs  across  the  Army,  Navy,  Air 
Force,  and  SOCOM  that  should  not  have  been  included  because  they  did 
not  meet  our  criteria  for  current  programs.16  For  example,  according  to  an 
Army  PEO,  90  of  140  programs  originally  reported  to  us  as  current  ACAT 
II  or  III  programs  by  the  Army  were  not  current  based  on  our  definition 
because  these  programs  had  delivered  more  than  90  percent  of  planned 
items  or  expended  more  than  90  percent  of  planned  funds.  Without  the 
consistent  identification  of  current  ACAT  II  and  III  programs,  DOD  and 
component  officials  do  not  know  the  accurate  number  of  these  programs 
and  may  miss  opportunities  to  identify  programs  that  may  need  more  or 
less  oversight  depending  on  whether  or  not  most  of  the  anticipated 
acquisition  funding  has  been  spent. 

Additionally,  component  guidance  for  defining  acquisition  programs  varies 
and  likely  resulted  in  inconsistent  reporting  across  components  of 
acquisitions  at  lower  dollar  values.  DOD  acquisition  policy  establishes  a 
cost  ceiling  and  cost  floor  for  ACAT  II  programs  and  a  cost  ceiling,  but  no 
cost  floor,  for  ACAT  III  programs.  However,  some  components  have 
supplemental  guidance  that  establishes  additional  acquisition  categories 
or  exclusions.  For  example,  SOCOM  and  Navy  have  guidance  that 
provides  for  certain  acquisitions  with  less  than  $10  million  in  total  RDT&E 
contracts,  less  than  $25  million  per  year  in  annual  procurement  funding, 
and  less  than  $50  million  total  in  procurement  contracts  to  be  categorized 
as  non-ACAT  programs.  Specifically,  SOCOM  designates  these  low  cost, 
schedule,  and  technical  risk  efforts  to  field  special  operations-peculiar 
capabilities  as  abbreviated  acquisition  projects.  Similarly,  the  Navy 
designates  lower  dollar  value  programs  that  do  not  require  operational 
testing  and  evaluation  as  abbreviated  acquisition  programs.17  Army  and 
Air  Force  policies  do  not  provide  for  lower  dollar  threshold  categories 


16We  identified  these  programs  using  information  available  in  the  data  and  through  follow 
up  with  PEOs.  This  is  a  minimum  number  of  programs  removed.  Because  PEOs  did  not 
provide  information  about  program  phase  for  all  programs,  there  may  also  be  other 
programs  reported  in  the  data  that  do  not  meet  our  definition  of  current. 

17According  to  Navy  acquisition  policy,  abbreviated  acquisition  programs  are  supposed  to 
be  captured  in  the  Navy’s  Acquisition  Program  list.  However,  acquisition  policy  officials 
told  us  they  do  not  have  insight  into  these  programs  at  the  component  level. 
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beneath  the  ACAT  III  level.  According  to  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics, 
acquisition  programs  at  these  dollar  levels  should  be  reported  as  ACAT  III 
programs.  However,  Navy  and  SOCOM  did  not  include  their  non-ACAT 
programs  in  the  ACAT  II  and  III  program  data  reported  to  us. 

There  was  also  variation  within  components  as  to  how  acquisition 
programs  were  categorized.  Army  and  Air  Force  acquisition  officials  told 
us  that  some  programs  that  should  have  been  considered  ACAT  II  or  III 
programs  in  accordance  with  component  acquisition  policy  were  not. 
These  officials  also  told  us  that  PEOs  may  have  counted  and  handled 
programs  differently  in  the  absence  of  a  clear  definition  of  what  should  be 
considered  a  program  of  record.  For  example,  Army  officials  told  us  that 
categorizing  information  technology  programs  was  sometimes 
challenging,  and  they  have  worked  with  PEOs  to  review  the 
categorization  of  certain  information  technology  programs.  Army  and  Air 
Force  officials  told  us  that  as  a  result  of  confusion  among  PEOs  about 
whether  or  not  certain  programs  should  be  considered  ACAT  II  or  III 
programs,  they  have  needed  to  add  and  remove  numerous  ACAT  II  and 
III  programs  from  component  information  systems  over  the  past  year. 

The  types  of  issues  we  identified  may  have  also  contributed  to 
components  reporting  varying  numbers  of  ACAT  II  and  III  programs  in 
response  to  different  requests  for  information  during  the  same  time  frame. 
Specifically,  concurrent  with  reporting  755  current  ACAT  II  and  III 
programs  to  us,  the  Army,  Navy,  and  Air  Force  reported  1,360  ACAT  II 
and  III  programs  in  a  presentation  to  the  DOD  Business  Senior 
Integration  Group,  which  is  chaired  by  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  and  oversees  DOD’s  Better 
Buying  Power  initiatives.18  Acquisition  officials  from  these  components 
told  us  they  were  unable  to  fully  explain  the  reasons  for  the  difference 
between  the  numbers  of  programs  reported. 


1 8 

DOD’s  Better  Buying  Power  initiative  2.0  requires  that  services  institute  a  system  to 
measure  the  cost  performance  of  programs  and  institutions  and  to  assess  the 
effectiveness  of  acquisition  policies.  As  part  of  that  effort  DOD  instructed  Component 
Acquisition  Executives  to  determine  how  to  best  measure  non-ACAT  I  performance  trends 
and  brief  the  results  to  the  DOD’s  Business  Senior  Integration  Group  by  November  1, 
2013. 
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Components  Have  Taken 
Steps  to  Improve  AC  AT  II 
and  III  Program  Data,  but 
Have  Not  Fully  Addressed 
the  Causes  of  the 
Reliability  Problems 


The  Army,  Navy,  Air  Force,  and  SOCOM  have  established  information 
systems  to  track  cost  and  schedule  data  for  ACAT  II  and  III  programs  and 
taken  some  steps  to  address  issues  related  to  the  completeness  and 
accuracy  of  information  tracked  in  these  systems.  CBDP  officials  told  us 
they  recognize  the  value  of  establishing  a  system  to  track  data  on  ACAT 
II  and  III  programs  and  are  determining  the  capabilities  that  would  be 
needed  in  such  a  system.  Specifics  of  component  efforts  follow: 

•  According  to  Navy  officials,  data  in  the  Navy’s  Research, 

Development  &  Acquisition  Information  System  has  potentially  been 
incomplete  because  ACAT  II  and  III  program  data  has  not  been 
consistently  entered  into  the  system.  The  Navy  issued  an  updated 
policy  in  August  2014  that  requires  input  of  programmatic  information 
into  the  system  for  all  ACAT  programs. 

•  All  Air  Force  ACAT  II  and  III  programs  have  been  required  to  enter 
cost  and  schedule  data  into  the  System  Metric  and  Reporting  Tool 
since  2012,  but  Air  Force  officials  told  us  that  not  all  programs  had 
complied  with  the  requirement.  They  told  us  in  June  2014  that  they 
had  an  ongoing  effort  to  review  ACAT  II  and  III  programs  in  the 
system,  including  assessing  whether  cost  and  schedule  data  has 
been  populated.  Further,  the  Air  Force  has  established  an  investment 
master  list  that  will  capture  all  programs  receiving  RDT&E  and 
procurement  funding. 

•  SOCOM  requires  that  all  ACAT  II  and  III  programs  enter  program  and 
cost  data  into  its  centralized  acquisition  portal  data  system,  however 
SOCOM  officials  told  us  some  program  managers  have  been  more 
diligent  than  others  in  populating  the  system.  These  officials  told  us 
that  data  from  the  portal  is  now  used  to  conduct  monthly  program 
reviews,  rather  than  having  the  program  prepare  briefing  slides,  as  a 
way  to  encourage  program  managers  to  populate  and  regularly 
update  the  system. 

Components  have  also  taken  steps  to  improve  the  consistency  with  which 
they  identify  current  ACAT  II  and  III  programs.  For  example,  officials  at 
four  of  the  five  components  told  us  they  are  exploring  ways  to  identify  the 
acquisition  phase — technology  maturation  and  risk  reduction,  engineering 
and  manufacturing  development,  production  and  deployment,  operations 
and  support — for  programs  in  component  information  systems,  which 
could  improve  their  ability  to  reliably  identify  current  programs.  The  Air 
Force  and  Army  have  also  made  efforts  to  address  concerns  they  had 
previously  identified  related  to  the  consistent  identification  of  ACAT  II  and 
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Ill  programs  within  their  components.  The  Air  Force  issued  guidance  in 
January  2014,  detailing  which  programs  are  and  are  not  considered  to  be 
acquisition  programs  and  acquisition  policy  officials  told  us  they  have 
been  meeting  with  individual  PEOs  to  clarify  any  misunderstandings 
about  how  acquisition  programs  should  be  categorized.  Officials  from  the 
Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics, 
and  Technology  also  told  us  they  are  in  the  process  of  revising  the  Army’s 
acquisition  guidance,  with  input  from  PEOs,  to  more  precisely  define  the 
types  of  programs  that  should  and  should  not  be  considered  to  be 
acquisition  programs. 

However,  the  components’  efforts  do  not  fully  address  the  accuracy, 
completeness,  and  consistency  issues  we  identified  with  ACAT  II  and  III 
program  data.  For  example,  the  components  have  not  established 
systematic  processes  to  perform  data  quality  tests  on  PEO-submitted 
data  and  assess  the  results  to  help  identify  problems,  such  as  basic  math 
errors  or  missing  data,  for  further  review.  These  types  of  tests  and 
assessments  can  be  an  important  step  in  determining  whether  data  can 
be  used  for  its  intended  purposes.19  Additionally,  the  components  have 
not  developed  plans  that  detail  how  they  will  implement  or  sustain  data 
improvement  efforts.  For  example,  the  components  have  not  developed 
implementation  steps  for  assessing  data  reliability  on  an  ongoing  basis  or 
metrics  to  assess  the  success  of  data  cleanup  efforts.  Developing  such 
plans  is  a  key  project  management  practice.20  Without  establishing  this 
planning  foundation,  the  components  will  not  be  in  a  sound  position  to 
effectively  monitor  and  evaluate  the  implementation  of  efforts  to  improve 
component  data.  Finally,  efforts  to  consistently  identify  current  ACAT  II 
and  III  programs  have  been  focused  within  individual  components. 

Without  a  consistent  understanding  about  which  programs  should  be 
considered  to  be  current  ACAT  II  or  III  programs  across  components, 
similar  programs  will  continue  to  be  reported  on  differently,  thereby 
limiting  the  consistency  and  comparability  of  ACAT  II  and  III  program  data 
across  DOD. 


19GAO,  Assessing  the  Reliability  of  Computer-Processed  Data,  GAO-09-680G 
(Washington,  D.C.:  July  2009).  Examples  of  data  quality  tests  include  testing  for  missing 
data,  either  entire  missing  records  or  missing  values  in  key  data  elements;  looking  for 
duplicate  records;  looking  for  invalid  or  duplicate  identifiers;  and,  examining  the 
relationship  of  one  data  element  to  another. 

on 

Project  Management  Institute,  Inc.  The  Standard  for  Project  Management,  Third  Edition, 
2013. 
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DOD  components  lack  consistent  cost  and  schedule  performance  metrics 
to  assess  performance  trends  across  ACAT  II  and  III  programs.  As  part  of 
the  department’s  Better  Buying  Power  initiatives,  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  instructed  DOD 
components  to  determine  how  best  to  measure  performance  trends  for 
non-ACAT  I  programs.  Federal  internal  control  standards  also  emphasize 
the  importance  of  comparing  actual  performance  to  planned  or  expected 
results  throughout  the  organization  to  help  ensure  effective  results  are 
achieved  and  actions  are  taken  to  address  risks.  The  Army,  Navy,  and  Air 
Force  briefed  DOD’s  Business  Senior  Integration  Group  in  November 
2013  on  their  current  efforts  and  plans  regarding  assessing  ACAT  II  and 
III  cost  and  schedule  performance,  but  no  specific  follow-on  actions  or 
action  plans  have  been  developed. 

Unlike  MDAPs  and  MAIS  programs,  ACAT  II  and  III  programs  are  not 
required  to  report  cost  and  schedule  data  in  a  consistent  fashion,  despite 
the  potential  benefits  of  such  reporting.  MDAP  and  MAIS  programs  are 
required  to  report  key  cost  and  schedule  metrics  to  Congress  in  a 
standardized  format  through  Selected  Acquisition  Reports  and  MAIS 
Annual  Reports,  respectively.  Cost  and  schedule  data  for  these  reports 
are  pulled  from  DOD’s  web-based  DAMIR  system.  MDAP  cost  and 
schedule  data  are  used  by  DOD  for  its  annual  assessment  of  the 
performance  of  the  defense  acquisition  system,  which  the  department 
uses  to  improve  acquisition  program  performance  and  inform  policy  and 
programmatic  decisions.  ACAT  II  and  III  programs  are  not  required  to 
produce  similar  cost  and  schedule  reporting  as  larger  programs  and  do 
not  have  to  provide  program  data  in  DAMIR.  According  to  officials  from 
the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics,  although  minor  adjustments  may  be  needed  for  reporting 
purposes,  there  is  nothing  that  prevents  components  from  using  DAMIR 
to  capture  data  on  ACAT  II  and  III  programs,  and  acquisition  officials  from 
the  Army  have  considered  using  it.  Additionally,  CBDP  officials  told  us 
they  would  consider  DAMIR  when  exploring  potential  systems  to  track 
ACAT  II  and  III  program  data. 

DOD  component  officials  told  us  they  are  not  yet  sure  how  best  to 
measure  cost  and  schedule  performance  across  ACAT  II  and  III 
programs.  For  example,  Army  officials  told  us  that  analysis  of  component¬ 
wide  ACAT  II  and  III  performance  trends  may  not  make  sense  given  the 
differences  across  programs.  Navy,  SOCOM,  Air  Force,  and  CBDP 
officials  told  us  they  are  interested  in  tracking  cost  and  schedule 
performance  trends  across  ACAT  II  or  III  programs,  but  are  still  working  to 
define  performance  metrics  and  address  limitations  in  existing  data  or 


DOD  Lacks  Consistent 
Cost  and  Schedule 
Performance  Metrics  and 
Faces  Additional 
Challenges  Measuring 
Trends 
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Data  Limitations  for  Measuring 
Cost  and  Schedule 
Performance  Trends 


reporting  capabilities.  For  example,  the  Air  Force  has  attempted  to  assess 
cost  and  schedule  performance  for  a  subset  of  ACAT  II  and  III  programs, 
but  acquisition  officials  noted  that  the  process  was  very  resource¬ 
intensive,  and  they  had  concerns  about  the  reliability  of  the  cost  and 
schedule  information  used  in  their  analysis  given  the  lack  of  variability  in 
program  performance  over  time.  While  the  components  have  developed 
oversight  mechanisms  to  review  individual  ACAT  II  and  III  program 
performance,  such  as  through  periodic  program  status  reviews  at  the 
PEO  level  and  program  or  portfolio  reviews  by  senior  component 
acquisition  officials,  without  assessing  performance  trends  across  ACAT 
II  and  III  programs,  DOD  and  its  components  may  be  missing 
opportunities  to  identify  and  analyze  differences  between  actual  and 
expected  performance  and  develop  strategies  to  address  related  risks 
throughout  the  department. 

When  we  analyzed  information  available  on  cost  and  schedule 
performance,  we  determined  that  we  could  not  assess  cost  performance 
for  139  programs  out  of  a  non-generalizable  sample  of  170  programs  and 
schedule  performance  for  105  of  the  170  programs.21  In  addition  to 
missing  or  misreported  cost  data,  we  identified  two  challenges  to 
measuring  cost  and  schedule  performance  trends  for  ACAT  II  and  III 
programs:  (1)  programs  without  available  APBs  and  (2)  a  lack  of 
consistent  and  comparable  key  schedule  milestones  across  programs. 
See  figure  2  for  a  summary  of  our  assessment  of  the  data  available  to 
measure  cost  and  schedule  performance  and  appendix  III  for  additional 
details. 


21We  analyzed  information  provided  by  DOD  components  and  in  APBs  for  a  non- 
generalizable  sample  of  170  programs.  Since  there  are  no  established  metrics  to  assess 
ACAT  II  and  III  programs’  performance  trends,  we  assessed  the  information  available  for 
these  programs  using  the  same  metrics  we  use  to  assess  major  weapon  systems 
performance  in  our  annual  review.  We  measured  cost  performance  by  comparing  current 
cost  estimates  to  cost  estimates  from  program  start  and  schedule  performance  by 
comparing  changes  in  cycle  time  between  program  start  and  initial  operational  capability 
event  dates  in  current  schedules  to  the  schedules  from  program  initiation. 
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Figure  2:  Assessment  of  Data  Available  to  Measure  Acquisition  Category  (ACAT)  II  and  III  Cost  and  Schedule  Performance  for 
Non-generalizable  Sample  of  170  Programs 


No  original  or  current 
acquisition  program 
baseline 


No  original  or  current 
acquisition  program 
baseline 


139 

Total  number  of  programs 
without  information  needed 
to  assess  cost  performance 


105 

Total  number  of  programs 
without  information 
needed  to  assess 
schedule  performance 


Source:  GAO  analysis  of  component-reported  ACAT  II  and  III  data  and  acquisition  program  baselines  for  a  non-generalizable  sample  of  170  ACAT  II  and  III  programs.  |  GAO-15-188 

Note:  Programs  categorized  as  “no  original  or  current  acquisition  program  baseline”  are  those  that 
were  missing  either  the  original  or  current  acquisition  program  baseline,  or  both. 


Programs  without  APBs  For  75  of  the  170  programs  that  we  examined  in  detail,  we  could  not 

assess  cost  or  schedule  performance  because  DOD  components  had  not 
developed,  or  did  not  provide,  an  original  APB,  a  current  one,  or  both. 

The  components  were  unable  to  provide  APBs  for  various  reasons,  such 
as  because  they  could  not  locate  the  original  or  an  APB  was  not 
developed  at  program  start  or  to  this  point  in  the  life  of  the  program.  APBs 
are  critical  management  tools  that  establish  how  systems  will  perform, 
when  they  will  be  delivered,  and  what  they  will  cost.  According  to  DOD 
acquisition  policy,  APBs  are  required  of  all  acquisition  programs,  and 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  officials  told  us  they  generally  expect  all  acquisition  programs  to 


Page  16 


GAO-15-188  DOD’s  ACAT  II  and  III  Programs 


have  one.22  The  first  APB  is  approved  by  the  milestone  decision  authority 
prior  to  entry  into  system  development  (Milestone  B),  or  at  program 
initiation,  whichever  occurs  later.  APBs  may  be  revised  at  the  time  of 
significant  program  decisions,  such  as  milestones,  or  as  a  result  of  major 
program  changes  or  breaches  to  cost,  schedule,  or  performance 
parameters.  Table  3  shows  the  number  of  missing  APBs  by  component  in 
our  sample. 


Table  3:  Acquisition  Category  (ACAT)  II  and  III  Programs  in  Sample  Missing  One  or  More  Acquisition  Program  Baselines 
(APB) 

Total 

Army 

Navy 

Air  Force 

U.S.  Special  Operations 
Command 

Chemical  and  Biological 
Defense  Program 

Missing  original  APB  only 

51 

9 

9 

25 

4 

4a 

Missing  current  APB  only 

1 

1 

0 

0 

0 

0 

Missing  both  original  and 
current  APBs 

23 

0 

0 

13 

9 

1 

Programs  providing  both  APBs 

95 

35 

28 

10 

13 

9 

Total  programs  in  sample 

170 

45 

37 

48 

26 

14 

Source:  GAO  analysis  of  Department  of  Defense  component-provided  APBs.  |  GAO-15-188 


Notes:  The  first  APB  is  approved  by  the  milestone  decision  authority  prior  to  entry  into  system 
development  (Milestone  B),  or  at  program  initiation,  whichever  occurs  later.  APBs  may  be  revised  at 
the  time  of  significant  program  decisions,  such  as  milestones,  or  as  a  result  of  major  program 
changes  or  breaches  to  cost,  schedule,  or  performance  parameters.  Components  were  unable  to 
provide  APBs  for  various  reasons,  such  as  because  they  could  not  locate  the  original  or  an  APB  was 
not  developed  at  program  start  or  so  far  in  the  life  of  the  program. 

aFor  three  of  the  four  Chemical  and  Biological  Defense  Program  programs  that  did  not  have  APBs  at 
program  start,  the  decision  to  develop  the  APB  after  program  start  is  documented  in  a  decision 
memoranda. 


As  shown  in  table  3,  Air  Force  and  SOCOM  programs  were  missing  both 
original  and  current  APBs  much  more  often  than  other  components  for 
our  sample.  While  we  observed  some  programs  at  Army,  Navy,  and 
CBDP  that  were  missing  APBs,  most  of  these  were  programs  that  started 


22lnterim  Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense 
Acquisition  System,  (Nov.  25,  2013).  The  final  version  of  DOD  Instruction  5000.02  was 
issued  on  January  7,  2015;  however,  for  purposes  of  this  report,  we  analyzed  the 
information  requirements  contained  in  the  Interim  DOD  Instruction  5000.02  dated 
November  25,  2013.  An  APB  summarizes  a  program’s  cost,  schedule,  and  performance 
parameters.  It  is  the  agreement  between  the  milestone  decision  authority,  the  program 
manager,  and  his  or  her  acquisition  chain  of  command  that  is  used  for  tracking  and 
reporting  for  the  life  of  the  program  or  program  increment. 
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in  2010  or  earlier  and  were  missing  an  APB  from  program  start  but  had 
later  developed  one.  Air  Force  officials  told  us  there  has  been 
misunderstanding  among  PEOs  about  APB  requirements  and  that  they 
are  working  with  PEOs  to  ensure  that  APBs  or  equivalent  documents  are 
developed  for  each  acquisition  program.  The  Air  Force  issued  guidance  in 
March  2012  establishing  common  processes  and  procedures  for  APB 
development  and  reporting.23  However,  when  we  conducted  our  analysis 
in  July  2014,  13  of  48  Air  Force  programs  we  reviewed  were  missing  both 
an  original  and  current  APB.24  SOCOM  was  missing  original  and  current 
APBs  for  9  of  26  programs  in  our  sample.  SOCOM  officials  told  us  that 
milestone  decision  authorities  can  waive  the  requirement  for  APBs,  but 
SOCOM  acquisition  policy  does  not  require  documentation  of  APB 
waivers.  In  October  2014,  the  SOCOM  Acquisition  Executive  directed 
PEOs  to  improve  APBs  for  all  acquisition  programs  to  facilitate  the  ability 
to  measure  cost,  schedule,  and  technical  performance.  If  the  Air  Force 
and  SOCOM  do  not  consistently  implement  revised  guidance  related  to 
setting  formal  cost  and  schedule  baselines,  they  will  be  limited  in  their 
ability  to  accurately  assess  ACAT  II  and  III  program  performance  and 
performance  trends.  These  limitations  could  also  affect  DOD’s  ability  to 
understand  ACAT  II  and  III  performance  across  components  and  identify 
the  root  causes  of  both  positive  and  negative  program  outcomes. 

Lack  of  Comparable  Schedule 

Milestones 


We  were  unable  to  measure  schedule  performance  for  30  of  the  95  ACAT 
II  and  III  programs  in  our  sample  that  had  both  original  and  current  APBs 
because  they  lacked  comparable  program  start  or  initial  operational 
capability  milestone  dates.  For  example,  the  APB  developed  at  program 
start  for  the  Air  Force’s  B-1  Vertical  Situation  Display  Upgrade — an  ACAT 
II  program —  included  dates  for  program  start  (Milestone  B)  and  the  start 
of  production  (Milestone  C),  but  did  not  include  an  initial  operational 
capability  date  or  equivalent  milestone.  The  initial  operational  capability 
date  is  important  for  measuring  schedule  performance  because  it 
specifies  when  the  warfighter  can  expect  a  system  to  be  fielded.  While 
DOD  acquisition  policy  generally  permits  milestone  decision  authorities  to 


oo 

Department  of  the  Air  Force,  Office  of  the  Assistant  Secretary  of  the  Air  Force  for 
Acquisition  Memo,  Subject:  Acquisition  Program  Baseline  (APB)  Standardization  and 
Reporting,  March  8,  2012. 

24At  the  Air  Force,  2  of  13  programs  we  reviewed  that  were  missing  both  original  and 
current  APBs  were  quick  reaction  capability  programs,  which  Air  Force  policy  exempts 
from  having  APBs.  Air  Force  Instruction  63-114,  Quick  Reaction  Capability  Process, 
January  4,  2011. 


Page  18 


GAO-15-188  DOD’s  ACAT  II  and  III  Programs 


tailor  program  milestones  in  the  interest  of  creating  effective  and  efficient 
programs,  the  lack  of  consistency  among  programs  with  regard  to  key 
schedule  milestones  could  hinder  DOD  and  component  efforts  to  develop 
effective  metrics  to  track  ACAT  II  and  III  schedule  performance.25 


ACATII  and  III 
Program 

Performance  Most 
Frequently  Affected 
by  Requirements 
Changes  and  Testing 
Issues 


Thirteen  of  the  15  ACAT  II  or  III  programs  we  reviewed  in-depth  had 
exceeded  the  cost  or  schedule  targets  in  their  original  APBs.  These 
programs  cited  changing  requirements,  testing  issues,  quantity  changes, 
and  flaws  in  original  cost  estimates,  among  other  factors,  as  the  reasons 
for  cost  and  schedule  growth.  The  programs  we  reviewed  cited  other 
factors,  such  as  a  reliance  on  mature  technology — including  commercial 
or  government  off-the-shelf  or  other  non-developmental  items — and  early 
involvement  of  stakeholders  or  users  as  contributing  to  reduced  risk  of 
cost  or  schedule  growth.  We  have  previously  reported  that  similar  factors 
affect  the  performance  of  DOD’s  MDAPs.  Appendix  IV  provides  additional 
details  about  the  programs  we  reviewed. 


Programs  Most  Frequently 
Attributed  Cost  and 
Schedule  Growth  to 
Requirements  Changes 


Thirteen  of  the  15  ACAT  II  or  III  programs  we  reviewed  in-depth  had 
exceeded  the  cost  or  schedule  targets  in  their  original  APBs.26  We  did  not 
attempt  to  quantify  the  extent  to  which  these  programs  had  exceeded 
cost  or  schedule  targets  due  to  overall  concerns  about  the  reliability  of 
ACAT  II  and  III  cost  and  schedule  data  and  because  not  all  of  these 
programs  had  developed  APBs  at  program  start.  These  programs  most 
frequently  attributed  cost  growth  or  schedule  delays  to  changing 
requirements.  Testing  issues,  quantity  changes,  and  flaws  in  original  cost 
estimates  were  also  cited  by  at  least  5  of  the  13  programs  as  contributing 


25According  to  DODI  Interim  5000.02,  milestone  decision  authorities  may  tailor  programs, 
to  include  eliminating  phases  and  combining  or  eliminating  milestones  and  decision 
points,  unless  constrained  by  statute. 

26Two  of  our  15  case  studies  were  selected  in  part  based  on  the  fact  that  they  had 
experienced  substantial  cost  growth.  The  other  13  case  study  programs  were  selected 
either  because  data  provided  by  DOD  components  indicated  they  were  the  largest  ACAT 
II  or  III  programs  within  the  component  based  on  total  estimated  acquisition  cost  (10 
programs)  or  because  they  were  part  of  a  family  of  systems  (3  programs).  In  the  instances 
in  which  we  selected  families  of  systems  for  review,  we  reviewed  multiple  ACAT  II  or  III 
programs  included  in  the  family  of  systems,  but  considered  the  group  of  systems  as  a 
single  program  for  the  purposes  of  our  analysis.  To  determine  whether  a  program  had 
exceeded  a  cost  or  schedule  target  in  its  original  APB,  we  analyzed  whether  the  program 
had  exceeded  a  cost  or  schedule  threshold  parameter  in  the  earliest  APB  provided  by  the 
program. 
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to  cost  growth  or  schedule  delays.  All  but  1  of  the  13  programs  cited 
multiple  causes  for  cost  growth  or  schedule  delays,  including  factors 
beyond  those  listed  in  table  4. 


Table  4:  Factors  Most  Frequently  Cited  by  Acquisition  Category  (ACAT)  II  and  III 
Case  Study  Programs  as  Contributing  to  Cost  Increases  or  Schedule  Delays 


Factor 

Programs  with  cost 
growth  and  schedule 
delays 
(n=10) 

Programs  with 
cost  growth  only 
(n=3) 

Total 

(n=13) 

Changing  requirements 

6 

3 

9 

Testing  issues 

7 

0 

7 

Quantity  increases 

2 

3 

5 

Flaws  with  original  cost 
estimates 

4 

1 

5 

Source:  GAO  analysis  of  program  documents  and  interviews  with  agency  officials  for  13  case  study  programs  that  had  exceeded  at 
least  one  cost  or  schedule  threshold  from  the  program’s  original  acquisition  program  baseline.  |  GAO-15-188 


Note:  Programs  could  cite  more  than  one  factor  as  contributing  to  cost  increases  or  schedule  delays. 


Requirements  changes  were  associated  with  cost  growth  or  schedule 
delays  by  at  least  one  program  at  each  of  the  five  components  in  our 
review.  According  to  program  officials,  programs  added  or  increased 
requirements  due  to  situations  such  as:  adding  capability  to  a  new 
platform  that  had  not  been  planned  for  when  the  original  requirements 
were  approved;  creating  additional  variants  to  meet  requirements  that 
emerged  after  the  original  requirements  were  approved;  or  making 
improvements  or  refinements  to  a  system  in  development  or  production 
as  a  result  of  changes  in  the  operational  environment,  including  new 
threats.  For  example,  officials  from  the  Army’s  Synthetic  Environment 
Core  program,  which  is  providing  the  Army  a  common  virtual  environment 
that  links  virtual  simulators  and  simulations  into  an  integrated  and 
interoperable  training  environment,  told  us  that  increasing  terrain 
database  requirements  to  meet  additional  training  needs  have  contributed 
to  program  cost  increases  significant  enough  to  require  the  program  to  be 
recategorized  from  an  ACAT  III  to  an  ACAT  II  program.  Program  officials 
stated  that  in  some  cases,  the  additional  requirements  have  been 
unrealistic  from  either  a  cost  or  technological  perspective,  but  that 
historically  there  had  not  been  an  effective  process  to  prioritize 
requirements  or  enforce  capability  tradeoffs.  Table  5  provides  additional 
examples  from  our  case  studies  of  factors  cited  by  program  offices  as 
contributing  to  cost  growth  or  schedule  delays. 
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Table  5:  Examples  of  Factors  Most  Frequently  Cited  by  Acquisition  Category  (ACAT)  II  and  III  Case  Study  Programs  as 
Contributing  to  Cost  Increases  or  Schedule  Delays 

Factor 

Example 

Changing  requirements 

Research  and  development  costs  for  the  Marine  Corps’  Light  Armored  Vehicle  Command  and 
Control  Upgrade  program — which  is  designed  to  improve  the  vehicle’s  ability  to  receive  and 
transmit  data;  share  information  among  computer  workstations;  and  utilize  more  reliable,  mobile, 
and  long  range  communications  capabilities — increased  from  $22.2  million  at  program  start  in 
March  2005  to  $29.7  million  at  the  start  of  production  in  November  2009.  The  program  also 
experienced  a  more  than  two-year  delay  in  delivering  initial  capability  to  the  warfighter. 

According  to  program  officials,  this  cost  increase  and  schedule  delay  was  due  in  part  to 
requirements  changes  to  enhance  vehicle  survivability  against  increased  threats  from  improvised 
explosive  devices  in  Iraq  and  Afghanistan.  The  additional  armor  needed  to  meet  these 
requirements  reduced  the  space  available  for  the  communications  suite,  which  resulted  in  it 
being  redesigned. 

Testing  issues 

Total  acquisition  costs  for  the  Chemical  and  Biological  Defense  Program’s  Joint  Biological  Point 
Detection  System,  a  joint  program  designed  to  detect  and  identify  biological  threats,  increased 
from  $31 3.2  million  at  development  start  in  December  1 996  to  $777.4  million  as  of  September 
2011.  The  program  also  experienced  an  approximately  2  year  delay  in  delivering  its  initial 
capability.  CBDP  officials  attributed  the  cost  growth  and  schedule  delays  in  part  to  DOD-directed 
changes  to  the  testing  strategy  after  the  program  started.  The  revised  strategy  required  the 
program  to  conduct  more  realistic  testing,  which  included  a  system-level  test  with  live  biological 
agents.  Since  the  program  was  one  of  the  first  to  conduct  this  type  of  testing,  more  time  and 
funding  than  originally  expected  was  needed  to  develop  test  facilities  and  methodologies  and 
award  related  contracts. 

Quantity  increases 

Procurement  costs  for  the  Navy’s  Nulka  Shipboard  Improvement  program,  a  rocket-launched, 
active  electronic  warfare  countermeasure  system  designed  to  defend  ships  against  anti-ship 
missiles,  increased  from  $434.9  million  dollars  at  production  start  in  January  1999  to  $962.0 
million  as  of  April  2009.  During  this  period,  planned  system  quantities  increased  by 
approximately  85  percent,  from  88  systems  at  production  start  to  163  systems  as  of  April  2009. 
Expected  quantities  for  decoy  cartridges  used  by  the  system  to  lure  anti-ship  missiles  away  from 
their  intended  targets  also  increased  from  896  to  1,288  during  the  same  period.  According  to 
program  officials,  increases  in  the  number  of  systems  installed  and  decoys  required  was  largely 
due  to  the  outfitting  of  additional  ship  classes  not  originally  anticipated  by  the  program. 

Flaws  in  the  original  cost  estimate 

Total  acquisition  costs  for  the  Air  Force’s  Haystack  Ultra-Wideband  Satellite  Imaging  Radar 
program,  which  improved  the  capability  of  an  existing  radar,  increased  from  an  estimated  $41 
million  at  development  start  in  June  2004  to  $1 08  million  as  of  April  2013.  Program  officials  told 
us  that  because  the  program  was  started  as  a  low-cost  modification,  an  independent  cost 
estimate  was  not  completed.  As  a  result,  the  program  office  relied  on  the  contractor’s  overly 
optimistic  cost  assumptions  and  failed  to  adequately  account  for  risk  and  uncertainty. 

Source:  GAO  analysis  of  program  documents  and  interviews  with  agency  officials  for  selected  case  study  programs  that  had  exceeded  at  least  one  cost  or  schedule  threshold  from  the  program’s  original 
acquisition  program  baseline.  |  GAO-15-188 


Note:  All  dollar  figures  are  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 


We  have  previously  reported  that  similar  factors  have  negatively  affected 
the  cost  and  schedule  performance  of  MDAPs.  DOD’s  weapons  system 
programs  often  enter  the  acquisition  process  without  a  full  understanding 
of  requirements,  and  we  have  reported  numerous  times  that  requirements 
changes  or  changes  to  designs  to  meet  requirements  are  factors  in  poor 
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cost  and  schedule  outcomes.27  Additionally,  in  part  due  to  high  levels  of 
uncertainty  about  requirements,  program  cost  estimates  and  their  related 
funding  needs  are  often  flawed.  For  example,  in  2008  we  assessed  cost 
estimates  for  20  MDAPs  and  found  that  the  estimates  were  too  low  in 
most  cases  and  that  in  some  programs,  cost  estimates  were  off  by  billions 
of  dollars.28  The  programs  we  reviewed  frequently  lacked  sufficient 
knowledge  and  detail  to  develop  sound  cost  estimates,  which  effectively 
set  programs  up  for  cost  growth  and  schedule  delays. 


Programs  Most  Frequently 
Cited  the  Use  of  Mature 
Technologies  and  Early 
Stakeholder  Involvement 
as  Factors  That  Helped  to 
Reduce  the  Risk  of  Cost 
or  Schedule  Growth 


Program  officials  for  the  ACAT  II  and  III  programs  we  reviewed  most 
frequently  cited  the  reliance  on  mature  technology — including  commercial 
or  government  off-the-shelf  or  other  non-developmental  items — and  early 
involvement  of  stakeholders  or  users  as  factors  that  helped  to  reduce  the 
risk  of  cost  or  schedule  growth.  Both  of  these  factors  were  cited  by  5  or 
more  of  the  15  ACAT  II  or  III  program  offices  we  reviewed.  In  some 
cases,  these  factors  were  cited  by  programs  that  experienced  cost  growth 
or  schedule  delays,  for  example,  because  one  of  these  factors  may  have 
helped  a  program  partially  recover  from  a  cost  or  schedule  breach  or 
keep  initial  program  costs  lower  or  schedules  shorter  than  otherwise 
would  be  expected. 


Reliance  on  existing  mature  technologies  was  a  relevant  factor  for  the  two 
programs  we  reviewed  that  did  not  report  cost  growth  or  schedule  delays, 
and  the  most  frequently  cited  factor  contributing  to  reduced  risk  of  cost  or 
schedule  growth  among  all  of  the  programs  we  reviewed.  The  two 
programs  we  reviewed  that  appeared  to  be  on  track  to  meet  original  cost 
and  schedule  targets — the  Army’s  5.56  millimeter  Enhanced  Performance 
Round  program  and  SOCOM’s  Nonstandard  Aviation  program — relied  on 
modified  commercial  off-the-shelf  equipment  or  modified  existing  military 
service  equipment  or  assets.  The  Army’s  5.56  millimeter  Enhanced 
Performance  Round  was  an  incremental  engineering  change  to  replace 
the  Army’s  general  purpose  5.56  millimeter  bullet  with  a  new  bullet 


27See,  for  example,  GAO,  Defense  Acquisitions:  Assessments  of  Selected  Weapon 
Programs,  GAO-14-340SP  (Washington,  D.C.:  Mar.  31, 2014)  and  Defense  Acquisitions: 
Assessments  of  Selected  Weapon  Programs,  GAO-1 3-294SP  (Washington,  D.C.:  Mar. 
28,2013). 

po 

GAO,  Defense  Acquisitions:  A  Knowledge-Based  Funding  Approach  Could  Improve 
Major  Weapon  System  Program  Outcomes,  GAO-08-619  (Washington,  D.C.:  July  2, 
2008). 
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design,  which  features  a  copper  slug  and  exposed  hardened  steel 
penetrator.  SOCOM’s  Nonstandard  Aviation  program  acquires,  modifies, 
fields,  and  sustains  commercial  aircraft  to  transport  special  operations 
forces.  The  use  of  mature  technologies  was  also  cited  as  contributing  to 
reduced  risk  of  cost  or  schedule  growth  by  6  of  the  13  other  programs  we 
reviewed.  For  example,  according  to  program  documentation  for  the  Air 
Force’s  F-15E  Radar  Modernization  Program,  the  program  planned  to 
leverage  existing  commercial  and  government  off-the-shelf  technology 
from  other  fighter  aircraft  radar  systems  and  the  maturity  of  these 
technologies  significantly  lowered  program  development  risk  and  costs. 

Early  stakeholder  or  user  involvement  was  cited  by  5  of  the  15  programs 
we  reviewed  as  contributing  to  reduced  risk  of  cost  or  schedule  growth, 
including  1  of  the  2  programs  that  did  not  experience  cost  growth  or 
schedule  delays.  For  example,  officials  with  the  Army’s  5.56  millimeter 
Enhanced  Performance  Round  program  noted  that  constant 
communication  with  all  stakeholders,  engineers,  testers,  and  contractors 
was  essential  and  a  key  success  factor  for  the  program.  Similarly, 
program  officials  for  CBDP’s  Dismounted  Reconnaissance  Sets,  Kits,  and 
Outfits  program — which  provides  protective  equipment  for  chemical, 
biological,  radiological,  or  nuclear  hazards — told  us  that  the  participation 
of  all  of  the  military  services  at  the  beginning  of  the  program  helped  to  the 
keep  program  cost  and  schedule  on  track.  According  to  program  officials, 
they  integrated  user  input  from  the  outset,  including  in  developing  the 
concept  of  operations,  which  reduced  the  number  of  later  requirements 
changes.  At  the  time  of  our  review,  the  program  was  on  track  to  meet  its 
original  schedule  targets.  The  program’s  unit  cost  also  decreased 
between  the  start  of  development  and  production. 

We  have  previously  reported  that  similar  factors  appear  to  positively 
affect  the  cost  and  schedule  performance  of  MDAPs.  For  example,  in 
2010,  we  reported  on  MDAPs  that  appeared  to  be  stable  and  on  track  to 
meet  their  original  cost  and  schedule  targets.  We  found  that  the  stable 
programs  we  reviewed  leveraged  mature  technologies  that  had  been 
demonstrated  to  work  in  relevant  or  realistic  environments,  and  either  did 
not  consider  immature  technologies  or  deferred  immature  technologies  to 
later  program  increments.29  We  also  reported  in  2012  that  early 


29GAO,  Defense  Acquisitions:  Strong  Leadership  Is  Key  to  Planning  and  Executing  Stable 
Weapon  Programs,  GAO-10-522  (Washington,  D.C.:  May  6,  2010). 
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stakeholder  involvement  in  pre-system  development  reviews  helped 
facilitate  trade-offs  among  cost,  schedule,  and  technical  performance 
requirements.  For  example,  by  involving  both  the  requirements  and 
acquisition  communities  in  these  reviews,  the  Army  was  able  to  identify 
trade-offs  that  reduced  the  projected  unit  costs  for  the  Joint  Light  Tactical 
Vehicle  without  impairing  operational  needs.30 


At  Least  Five  Current 
Programs  Are  Near  or 
Over  the  ACAT  I  Cost 
Threshold,  but  Data 
Limitations  Hinder 
DOD’s  Ability  to 
Identify  Additional 
Programs 


Data  provided  by  DOD  components  indicated  that  at  least  five  current 
ACAT  II  programs  were  approaching  or  had  exceeded  ACAT  I  cost 
thresholds  as  of  November  2013,  though  DOD  component  officials  told  us 
that  most  were  not  expected  to  become  MDAPs.31  We  were  unable  to 
identify  with  certainty  the  number  of  programs  likely  to  become  MDAPs 
because  of  data  reliability  issues  related  to  identifying  the  population  of 
ACAT  II  and  III  programs  and  their  estimated  cost.  Using  the  836 
programs  initially  reported  by  DOD  components  as  our  starting  point,  we 
identified  two  current  ACAT  II  programs  that  exceeded  the  ACAT  I 
threshold  for  RDT&E — $480  million  in  fiscal  year  2014  constant  dollars — 
and  three  current  ACAT  II  programs  that  were  within  10  percent  of  the 
ACAT  I  RDT&E  or  procurement  threshold — $2.79  billion  in  fiscal  year 
2014  constant  dollars — as  of  November  2013.  Of  these  five  programs, 
DOD  component  officials  told  us  that  four  would  not  become  MDAPs 
because,  for  example,  they  did  not  expect  further  program  cost  growth  or 
were  considering  restructuring  the  program,  and  that  component-level 
discussions  were  underway  with  regard  to  the  status  of  the  remaining 
program  (see  table  6). 


30GAO,  Weapons  Acquisition  Reform:  Reform  Act  Is  Helping  DOD  Acquisition  Programs 
Reduce  Risk,  but  Implementation  Challenges  Remain,  GAO-13-103  (Washington,  D.C.: 
Dec.  14,  2012). 

31The  Army’s  Kiowa  Warrior — Cockpit  and  Sensor  Upgrade  Program  was  also  within  10 
percent  of  the  ACAT  I  RDT&E  threshold  as  of  November  2013,  but  the  program  was 
terminated  in  March  2014.  As  a  result,  we  excluded  this  program  from  our  analysis. 
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Table  6:  Status  of  Current  Acquisition  Category  (ACAT)  II  or  III  Programs  Approaching  or  Exceeding  ACAT  1  Cost  Thresholds 
as  of  November  2013 

Department  of 

Defense 

component 

Program 

Cost  status 

Whether  component 
expects  program  to 
become  a  major  defense 
acquisition  program 

Component  rationale 

Army 

AN/TPQ-53 

Counterfire  Target 
Acquisition  Radar 
System 

Within  10  percent 
of  ACAT  1 
procurement 
threshold 

No 

The  Army  does  not  expect  the 
program’s  cost  to  exceed  the  ACAT 

1  threshold  based  on  current 
program  office  estimates. 

Army 

Close  Combat  Tactical 
Trainer 

Within  10  percent 
of  ACAT  1  research, 
development,  test, 
and  evaluation 
(RDT&E)  threshold 

No 

The  Army  does  not  expect  the 
program’s  cost  to  exceed  the  ACAT 

1  threshold  based  on  current 
program  office  estimates. 

Air  Force 

Engine 

Component 
Improvement  Program 

Exceeds  ACAT  1 
RDT&E  threshold 

No 

Air  Force  acquisition  officials  told  us 
they  are  working  with  the  program 
executive  officer  to  determine  the 
appropriate  structure  for  this  effort, 
which  consists  of  multiple  distinct 
research  and  development  efforts 
for  in-service  Air  Force  engines. 

Navy 

Advanced 

Arresting  Gear 

Exceeds  ACAT  1 
RDT&E  threshold 

Undetermined 

The  Navy  is  revising  the  program’s 
acquisition  program  baseline.  The 
Navy  will  submit  a  request  to 
change  the  acquisition  category,  if 
needed,  after  the  revised  baseline  is 
complete. 

Navy 

AN/SPY-1  D(V)  Radar 
Upgrade 

Within  10  percent 
of  ACAT  1  RDT&E 
threshold 

No 

The  Navy  does  not  plan  to  expend 
additional  RDT&E  funding  on  the 
program.  Therefore,  the  program’s 
cost  is  not  expected  to  exceed  the 
ACAT  1  threshold. 

Source:  GAO  analysis  of  information  provided  by  Department  of  Defense  components.  |  GAO-15-188 


We  also  found  that  the  Army,  Navy,  and  Air  Force  did  not  comply  with 
DOD  reporting  requirements  for  programs  approaching  ACAT  I  cost 
thresholds.  DOD  acquisition  policy  was  revised  in  November  2013  to 
require  components  to  notify  the  Defense  Acquisition  Executive — the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics — 
when  an  increase  or  estimated  increase  in  program  cost  or  a  change  in 
acquisition  strategy  will  result  in  a  possible  reclassification  of  a  formerly 
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lower  acquisition  category  program  as  an  ACAT  I  program.32  Notification 
is  to  be  made  as  soon  as  the  DOD  component  anticipates  that  the 
program’s  cost  is  within  10  percent  of  the  minimum  cost  threshold  of  the 
next  acquisition  category  level.  Policy  officials  from  the  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
told  us  that  they  expect  components  to  provide  notification  upon  an  initial 
determination  that  the  program’s  estimated  cost  is  within  10  percent  of 
either  ACAT  I  cost  threshold  to  facilitate  the  involvement  of  appropriate 
stakeholders  in  discussions  about  the  future  of  the  program.  However, 
DOD  components  had  not  notified  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  about  the  five  programs  we 
identified  as  exceeding  or  approaching  an  ACAT  I  cost  threshold.  As  of 
November  2014,  the  Army  had  drafted  but  not  yet  sent  a  notification 
memorandum  for  the  AN/TPQ-53  Counterfire  Target  Acquisition  Radar 
System — and  that  notification  process  was  triggered  as  a  result  of  our 
review.  Component  officials  told  us  they  had  not  notified  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  about  the 
remaining  four  programs  either  because  the  program  was  not  expected  to 
exceed  an  ACAT  I  cost  threshold  or  because  component-level 
discussions  about  the  status  of  the  program  were  still  ongoing. 

Further,  we  found  that  DOD  components’  ability  to  identify  additional 
ACAT  II  or  III  programs  that  may  be  approaching  or  exceeding  an  ACAT  I 
cost  threshold  is  hindered  by  a  lack  of  reliable  data  on  current  ACAT  II 
and  III  programs  and  their  estimated  costs.  For  example,  Army  officials 
were  unaware  that  the  Army’s  AN/TPG-53  Counterfire  Target  Acquisition 
Radar  System  was  within  10  percent  of  the  ACAT  I  procurement 
threshold  until  our  review  because  the  cost  estimate  reported  during 
program  reviews  had  historically  excluded  certain  costs  that  the  PEO  later 
determined  should  be  included.  Based  on  the  data  we  originally  received 
from  DOD  components,  we  initially  identified  23  programs  that  appeared 
to  be  within  10  percent  of  or  over  the  ACAT  I  RDT&E  or  procurement 
threshold.  We  later  determined  that  most  of  the  programs  were  not 


op 

Interim  Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense 
Acquisition  System,  enc.  1,  para.  2.c(1),  at  40  (Nov.  25,  2013).  Prior  to  November  2013, 
DOD  acquisition  policy  required  that  components  notify  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  when  ACAT  II  or  III  programs  were  reclassified 
as  ACAT  I  programs.  Department  of  Defense  Instruction  5000.02,  Operation  of  the 
Defense  Acquisition  System,  enc.  3,  para.  I.b,  at  32  (Dec.  8,  2008). 
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current  programs,  because  for  example,  they  were  in  sustainment,  or 
reported  cost  data  incorrectly  (see  fig.  3  below). 


Figure  3:  Analysis  of  Acquisition  Category  (ACAT)  II  or  III  Programs  Initially 
Reported  by  Department  of  Defense  (DOD)  Components  with  Estimated  Costs 
Approaching  or  Exceeding  ACAT  I  Cost  Thresholds 


Number  of  programs 
18 


RDT8.E 


11 


<j  Not  a  current  ACAT  II  or  III  program 


^  Incorrectly  reported  cost  estimate 


^  Current  ACAT  II  or  III  program  approaching  or  exceeding  ACAT  I  threshold 


Procurement 


Applicable  cost  threshold 


Source:  GAO  analysis  of  DOD  component  data.  |  GAO-15-188 


Note:  RDT&E  stands  for  research,  development,  test,  and  evaluation. 


Without  access  to  reliable  data  on  current  ACAT  II  and  III  programs  and 
component-level  policies  to  help  ensure  compliance  with  reporting 
requirements,  components  will  be  at  risk  of  failing  to  identify  programs 
that  may  become  MDAPs.  Potential  delays  in  identifying  these  programs 
and  notifying  decision  makers  directly  affects  the  visibility  that  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  and  other 
key  stakeholders,  including  Congress,  have  into  them,  as  well  as  their 
ability  to  exercise  oversight  over  programs  that  are  some  of  the 
department’s  largest  investments. 


Conclusions 


DOD  weapon  system  acquisition  represents  one  of  the  largest  areas  of 
the  government’s  discretionary  spending,  but  much  of  this  spending  is  still 
not  well  understood.  DOD’s  primary  focus  has  been  on  overseeing  and 
assessing  the  performance  of  its  large  ACAT  I  major  defense  acquisition 
programs,  but  the  annual  funding  spent  on  ACAT  II  and  III  acquisition 
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programs  may  be  just  as  significant.  Yet  data  provided  by  DOD 
components  were  so  unreliable  that  we  were  unable  to  accurately  identify 
even  a  minimum  number  or  total  cost  of  DOD’s  ACAT  II  and  III  programs. 
While  tailoring  documentation  and  reporting  requirements  for  “smaller” 
programs  can  be  a  reasonable  approach  to  help  prioritize  limited 
oversight  resources,  if  DOD  and  its  components  are  to  effectively  manage 
their  investment  dollars,  they  must  be  able  to  account  for  how  they  are 
spending  their  money  and  how  well  they  are  spending  it  on  the  full  range 
of  acquisition  programs.  Having  timely  and  reliable  data  on  smaller 
acquisition  programs  is  also  critical  for  providing  effective  oversight  and 
bringing  the  right  oversight  resources  to  bear,  when  needed,  to  make 
sure  troubled  smaller  programs  do  not  grow  into  major  ones. 

The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  has  recognized  the  value  of  having  good  data  on  DOD’s 
acquisition  programs — including  its  ACAT  II  and  III  programs — to  assess 
the  performance  of  the  defense  acquisition  system  and  identify  the  factors 
that  affect  program  performance.  But  work  remains  to  make  sure 
information  on  the  complete  range  of  DOD  acquisition  programs  is 
consistently  available.  DOD  components  have  taken  and  continue  to  take 
steps  to  improve  the  reliability  of  ACAT  II  and  III  program  data,  but  they 
do  not  fully  address  the  limitations  we  identified — missing  data, 
widespread  data  entry  issues  and  inconsistent  reporting — or  the  causes 
of  these  issues,  including:  the  lack  of  a  common  definition  of  a  current 
acquisition  program;  insufficient  data  reliability  testing;  and  inconsistent 
compliance  with  requirements  for  acquisition  program  baselines  and 
reporting  on  ACAT  II  and  III  programs  that  may  become  major  programs 
due  to  cost  growth.  Components  also  lacked  plans  to  ensure  their 
intended  actions  are  implemented  and  improvements  to  data  collection 
and  analysis  are  sustained  over  the  long  term.  Until  these  limitations  are 
addressed,  DOD  components  will  be  unable  to  generate  reliable 
information  to  effectively  manage  and  oversee  their  ACAT  II  and  III 
programs. 


Recommendations  for  are  ma*<'n9  f°ur  recommendations  to  improve  DOD’s  ability  to  collect 

and  maintain  reliable  data  on  its  acquisitions.  Specifically,  we  recommend 
Executive  Action  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of  Defense  for 

Acquisition,  Technology,  and  Logistics,  in  consultation  with  DOD 
components,  to  take  the  following  actions: 

•  establish  guidelines  on  what  constitutes  a  “current”  ACAT  II  or  III 
program  for  reporting  purposes;  the  types  of  programs,  if  any,  that  do 
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not  require  ACAT  designations;  and  whether  the  rules  for  identifying 
current  MDAPs  would  be  appropriate  for  ACAT  II  and  III  programs; 
and 

•  determine  what  metrics  should  be  used  and  what  data  should  be 
collected  on  ACAT  II  and  III  programs  to  measure  cost  and  schedule 
performance;  and  whether  the  use  of  DAMIR  and  the  MDAP  selected 
acquisition  report  format  may  be  appropriate  for  collecting  data  on 
ACAT  II  and  III  programs. 

We  also  recommend  that  the  Secretary  of  Defense  direct  the  Secretaries 
of  the  Air  Force,  Army,  and  Navy  and  the  Commander  of  SOCOM  to  take 
the  following  actions: 

•  assess  the  reliability  of  data  collected  on  ACAT  II  and  III  programs 
and  work  with  PEOs  to  develop  a  strategy  to  improve  procedures  for 
the  entry  and  maintenance  of  data;  and 

•  develop  implementation  plans  to  coordinate  and  execute  component 
initiatives  to  improve  data  on  ACAT  II  and  III  programs. 

We  are  also  making  two  recommendations  to  help  ensure  compliance 
with  relevant  provisions  of  DOD  acquisition  policy  with  the  purpose  of 
improving  DOD’s  ability  to  provide  oversight  for  ACAT  II  and  III  programs, 
including  those  programs  that  may  become  MDAPs. 

•  We  recommend  that  the  Secretary  of  Defense  direct  the  Secretary  of 
the  Air  Force  and  Commander  of  SOCOM  to  establish  a  mechanism 
to  ensure  compliance  with  APB  requirements  in  DOD  policy. 

•  We  recommend  that  the  Secretary  of  Defense  direct  the  Secretaries 
of  the  Air  Force,  Army,  and  Navy  to  improve  component  procedures 
for  notifying  the  Defense  Acquisition  Executive  of  programs  with  a 
cost  estimate  within  10  percent  of  ACAT  I  cost  thresholds. 


Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  DOD  for  review  and  comment.  In  its 
written  comments,  which  are  reprinted  in  full  in  appendix  V,  DOD  partially 
concurred  with  all  six  of  our  recommendations.  However,  as  discussed 
below,  it  is  unclear  whether  the  actions  that  DOD  plans  to  take  will  fully 
address  the  issues  we  raised  in  this  report. 

DOD  partially  concurred  with  our  first  recommendation  to  establish 
guidelines  on  what  constitutes  a  “current”  ACAT  II  or  III  program  for 
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reporting  purposes;  the  types  of  programs,  if  any,  that  do  not  require 
ACAT  designations;  and  whether  the  rules  for  identifying  current  MDAPs 
would  be  appropriate  for  ACAT  II  and  III  programs.  DOD  also  partially 
concurred  with  our  second  recommendation  related  to  determining  what 
metrics  should  be  used  and  what  data  should  be  collected  on  ACAT  II 
and  III  programs  to  measure  cost  and  schedule  performance.  In  its 
response,  DOD  stated  that  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  would  review  the  existing  policy 
direction  for  ACAT  II  and  III  programs  to  determine  whether  it  needs  to  be 
altered  or  supplemented  to  facilitate  data  collection  or  reporting.  However, 
as  our  review  found,  the  question  is  not  whether  policy  needs  to  be 
revised,  but  how  it  needs  to  be  revised.  We  found  that  the  existing  policy 
direction  was  not  adequate  to  ensure  consistent  data  collection  and 
reporting  on  ACAT  II  and  III  programs  or  their  cost  and  schedule 
performance  and  our  recommendations  were  designed  to  address  those 
issues.  We  continue  to  believe  that  additional  guidelines  for  components 
regarding  which  programs  should  be  considered  current  ACAT  II  and  III 
programs  for  reporting  purposes  and  consistent  metrics  to  measure 
performance  trends,  among  other  actions,  are  needed  to  correct  the 
issues  we  found. 

DOD  partially  concurred  with  our  third  and  fourth  recommendations  to 
assess  the  reliability  of  data  collected  on  ACAT  II  and  III  programs  and 
work  with  PEOs  to  develop  a  strategy  to  improve  procedures  for  the  entry 
and  maintenance  of  data;  and  develop  implementation  plans  to 
coordinate  and  execute  component  initiatives  to  improve  data  on  ACAT  II 
and  III  programs,  respectively.  In  its  response,  DOD  stated  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  will  direct 
the  DOD  components  to  evaluate  the  data  collected  on  ACAT  II  and  III 
programs  and  report  back  to  him  on  their  assessment  of  the  reliability  of 
that  data  and  the  status  of  the  plans  to  improve  the  availability  and  quality 
of  the  data.  DOD’s  response  represents  a  good  first  step  towards 
assessing  the  reliability  of  its  ACAT  II  and  III  program  data,  but  the 
response  does  not  fully  address  our  recommendations.  DOD’s  response 
does  not  address  whether  components  would  be  required  to  develop 
strategies  with  PEOs  to  improve  the  entry  and  maintenance  of  data,  as 
we  recommended.  We  continue  to  believe  that  developing  these 
strategies  with  those  responsible  for  entering  and  maintaining  program 
data  on  a  day-to-day  basis,  including  PEOs,  is  important  to  make  sure  the 
causes  of  DOD’s  data  quality  problems  are  fully  understood  and 
addressed  in  a  manner  that  can  be  implemented.  Further,  DOD’s 
response  does  not  directly  address  our  recommendation  to  develop 
implementation  plans  for  component  efforts.  We  believe  that  fully 
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implementing  this  recommendation  is  essential  for  ensuring  that  DOD  and 
its  components  can  effectively  monitor  and  evaluate  the  implementation 
of  component  initiatives  to  improve  ACAT  II  and  III  data. 

DOD  partially  concurred  with  our  fifth  recommendation  to  direct  the 
Secretary  of  the  Air  Force  and  Commander  of  SOCOM  to  establish  a 
mechanism  to  ensure  compliance  with  APB  requirements  in  DOD  policy. 
DOD  also  partially  concurred  with  our  sixth  recommendation  to  direct  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy  to  improve  component 
procedures  for  notifying  the  Defense  Acquisition  Executive  of  programs 
with  a  cost  estimate  within  10  percent  of  ACAT  I  cost  thresholds.  In  its 
response,  DOD  stated  that  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  will  issue  guidance  to  DOD 
components  reiterating  the  APB  requirements  for  ACAT  II  and  III 
programs  and  directing  that  the  Defense  Acquisition  Executive  be  notified 
when  an  increase  or  estimated  increase  in  program  cost  is  within  10 
percent  of  the  ACAT  I  cost  thresholds.  Reiterating  existing  departmental 
policy  on  these  issues  may  help  raise  awareness  at  the  component  level, 
but  without  additional  enforcement  mechanisms  it  may  not  address  the 
causes  of  the  deficiencies  we  discuss  in  this  report.  For  example,  the  Air 
Force  has  issued  component-level  guidance  directing  the  development  of 
APBs.  However,  we  found  that  programs  were  not  in  compliance  with  the 
guidance,  which  demonstrates  the  need  to  improve  enforcement 
mechanisms,  such  as  ensuring  milestone  decision  authorities  do  not 
approve  programs  to  proceed  through  acquisition  milestones  without 
APBs.  Similarly,  with  regard  to  our  recommendation  on  notification 
requirements  for  programs  approaching  the  ACAT  I  threshold,  we  found 
that  component  officials  cited  reasons  other  than  a  lack  of  awareness  of 
the  policy  for  not  notifying  the  Defense  Acquisition  Executive  of  these 
programs’  cost  growth.  As  a  result,  we  continue  to  believe  that  DOD 
should  fully  implement  our  recommendation  by  directing  components  to 
improve  their  notification  procedures. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics;  the  Secretaries  of  the  Army, 
Navy,  and  Air  Force;  the  Commander  of  U.S.  Special  Operations 
Command;  the  Assistant  Secretary  of  Defense  for  Nuclear,  Chemical,  and 
Biological  Defense  Programs;  and  other  interested  parties.  This  report  will 
also  be  available  at  no  charge  on  GAO’s  website  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  by  e-mail  at  sullivanm@gao.gov. 
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Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
Affairs  may  be  found  on  the  last  page  of  this  report.  Key  contributors  to 
this  report  are  listed  in  appendix  VI. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


Our  objectives  were  to  assess  (1)  the  extent  to  which  information  is 
available  on  the  number  of  the  Department  of  Defense’s  (DOD)  current 
acquisition  category  (ACAT)  II  and  III  programs,  their  total  estimated 
acquisition  cost,  and  cost  and  schedule  performance;  (2)  the  factors 
affecting  the  cost  and  schedule  performance  of  selected  ACAT  II  and  III 
programs;  and  (3)  the  number  of  current  ACAT  II  and  III  programs  that 
are  likely  to  become  major  defense  acquisition  programs  (MDAP). 

To  address  our  first  objective,  we  used  a  data  collection  instrument  (DCI) 
to  identify  and  collect  data  on  the  number  and  cost  of  current  ACAT  II  and 
III  programs  from  five  DOD  components  that  accounted  for  approximately 
88  percent  of  DOD’s  requested  research,  development,  test,  and 
evaluation  (RDT&E)  and  procurement  funding  in  the  President’s  Fiscal 
Year  2014  budget  request:  Army,  Air  Force,  Navy,  U.S.  Special 
Operations  Command  (SOCOM),  and  DOD’s  Chemical  and  Biological 
Defense  Program  (CBDP).1  We  used  a  DCI  to  obtain  ACAT  II  and  III 
program  data  based  on  preliminary  discussions  with  DOD  and  component 
officials  that  a  DCI  would  be  the  best  way  to  collect  the  information  of 
interest.  We  requested  that  each  component  identify  all  of  its  current 
ACAT  II  and  III  programs  and  provide  cost  data  and  descriptive 
information  for  each  program.  For  the  purposes  of  this  report,  we  defined 
a  current  program  as  one  that  has  been  formally  initiated  in  the 
acquisition  process  but  has  not  yet  delivered  90  percent  of  its  planned 
units  or  expended  90  percent  of  its  planned  expenditures.  For  cost  data, 
we  requested  components  provide  baseline  and  current  program 
estimates  in  millions  of  base  year  dollars,  to  include  estimates  for 
RDT&E,  procurement,  acquisition  operation  and  maintenance,  and 
military  construction,  as  well  as  the  program’s  total  acquisition  cost 
estimate  and  the  base  year  associated  with  the  estimate.  We  also 
collected  pertinent  information  for  each  program  including  program  name, 
ACAT  level,  type  of  acquisition  (automated  information  system  or  non- 


^avy  guidance  allows  for  programs  to  be  categorized  as  ACAT  IV  programs.  For  the 
purposes  of  this  report,  we  use  the  term  ACAT  III  to  also  include  Navy  and  Marine  Corps 
ACAT  IV  acquisition  programs. 
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automated  information  system),  milestone  decision  authority,  lead  DOD 
component,  and  program  executive  office.2 

To  obtain  additional  information  on  schedule  performance,  we  collected 
and  analyzed  acquisition  program  baseline  (APB)  documents,  which 
contain  program  schedule  and  cost  parameters,  for  a  non-generalizable 
random  sample  of  170  non-automated  information  system  ACAT  II  and  III 
programs.  To  select  the  programs,  we  used  the  initial  data  provided  to  us 
by  DOD  components  that  included  836  reported  ACAT  II  or  III  programs 
as  a  starting  point.  We  adjusted  our  selection  as  appropriate  to  account 
for  known  errors  in  the  data  at  the  time  of  selection  in  May  2014,  such  as 
programs  that  were  known  to  not  be  current  ACAT  II  or  III  programs.  Our 
intention  was  to  select  a  sample  that  would  be  generalizable  to  the 
population  of  current  ACAT  II  and  III  programs.  However,  after  selecting 
our  sample  we  determined  through  our  data  reliability  assessment  that 
the  population  of  current  ACAT  II  and  III  programs  could  not  be  reliably 
determined  and  that  our  sample  would  therefore  be  non-generalizable.3 
As  such,  results  of  this  analysis  cannot  be  used  to  make  inferences  about 
all  current  ACAT  II  and  III  programs.  When  APB  documents  were 
available  for  programs  in  our  sample,  we  reviewed  them  to  determine 
whether  they  contained  comparable  program  start  and  initial  operational 
capability  milestones  to  allow  us  to  measure  program  schedule 
performance.  We  also  used  the  APBs  collected  from  this  sample  of 
programs  as  part  of  our  reliability  assessment  of  ACAT  II  and  III  cost  data 
provided  by  DOD  components. 

Our  observations  on  DOD’s  ACAT  II  and  III  program  data  are  based  on 
the  original  data  submitted  by  the  components.  We  did  not  assess  the 
reliability  of  any  underlying  data  systems  that  may  have  been  used  to 
generate  this  information.  We  analyzed  the  original  data  provided  by  the 
components  because  it  reflects  the  information  DOD  would  have  had  on 


2We  requested  that  the  baseline  cost  estimate  reflect  the  objective  value  from  the 
program’s  original  acquisition  program  baseline.  We  requested  that  the  current  cost 
estimate  reflect  the  President’s  Fiscal  Year  2014  budget  request.  Our  instructions 
requested  an  explanation  for  any  field  the  component  was  unable  to  provide  and  that  zero 
be  entered  for  any  cost  estimate  element  without  a  value.  We  excluded  classified 
programs  from  the  scope  of  our  review. 

Consequently,  we  did  not  replace  14  programs  in  our  sample  that  we  determined  did  not 
meet  our  criteria  for  current  ACAT  II  and  III  programs  after  selection  and  instead  reduced 
the  sample  from  184  programs  to  170  programs. 
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ACAT  II  and  III  programs  at  the  time  we  collected  data.  From  January 
through  July  2014,  we  worked  with  components  to  attempt  to  correct 
problems  we  identified  in  the  data.  However,  we  continued  to  identify 
additional  errors.  As  a  result,  we  determined  that  the  data  provided  by 
DOD  components  in  response  to  our  DCI  were  not  sufficiently  reliable  to 
identify  the  number  of  current  ACAT  II  and  III  programs,  their  estimated 
acquisition  cost,  or  the  cost  performance  of  DOD’s  ACAT  II  and  III 
programs.  Appendix  III  contains  a  more  detailed  discussion  of  our  data 
reliability  assessment.  We  also  determined  that  we  could  not  assess 
schedule  performance  for  ACAT  II  and  III  programs  because  more  than 
half  of  programs  we  reviewed  in  our  sample  of  170  programs  were 
missing  source  documents  or  lacked  comparable  schedule  milestones. 

To  address  our  second  objective,  we  selected  a  non-generalizable 
sample  of  15  programs  from  the  data  provided  by  DOD  in  response  to  our 
DCI.4  We  selected  3  programs  from  each  component  included  in  our 
review.  For  each  component,  these  programs  were  selected  to  include 
the  largest  current  non-automated  information  system  ACAT  II  and  III 
program  based  on  total  acquisition  cost  as  of  the  President’s  Fiscal  Year 
2014  budget  submission  and  one  additional  program  based  on  factors 
such  as  significant  cost  growth  or  whether  the  program  was  part  of  a 
family  of  systems,  which  we  defined  as  a  related  group  of  programs 
consisting  of  multiple  increments  or  fielding  similar  capabilities  for  multiple 
platforms.5  To  select  the  programs,  we  used  the  initial  data  provided  to  us 
by  DOD  components  that  included  836  reported  ACAT  II  or  III  programs 
as  a  starting  point.  Programs  that  lacked  data  for  current  acquisition  cost, 
commodity  type,  or  ACAT  level  were  excluded  from  selection.  We  also 
adjusted  our  selection  as  appropriate  to  account  for  known  errors  in  the 
data  at  the  time  of  selection,  such  as  incorrectly-reported  cost  estimates, 
or  programs  that  were  known  to  not  be  current  ACAT  II  or  III  programs. 
However,  after  our  selection  we  identified  additional  concerns  with  the 


4This  sample  of  15  programs  was  selected  separately  from  the  sample  of  170  programs 
mentioned  above  using  different  criteria,  although  7  of  the  15  programs  are  included  in 
both  samples. 

5We  excluded  automated  information  systems  at  all  components  and  vaccine  programs  at 
CBDP  from  our  case  studies  because  our  focus  was  on  the  performance  of  weapon 
system  programs.  In  the  three  instances  in  which  we  selected  families  of  systems  for 
review — at  Navy,  SOCOM,  and  CBDP — we  reviewed  multiple  ACAT  II  or  III  programs 
included  in  the  family  of  systems,  but  considered  the  group  of  systems  as  a  single 
program  for  the  purposes  of  our  analysis. 
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data  reported  by  DOD  that  would  likely  have  changed  the  results  of  our 
selection  of  the  largest  ACAT  II  or  III  programs  at  certain  components. 

We  did  not  make  any  subsequent  adjustments  to  our  original  selection 
because  we  determined  that  the  data  provided  by  DOD  was  not 
sufficiently  reliable  to  enable  us  to  determine  the  largest  ACAT  II  or  III 
program  at  each  component. 

For  each  program,  we  analyzed  key  program  documents,  such  as  APBs, 
program  status  reports,  acquisition  strategies,  acquisition  decision 
memoranda,  and  requirements  documentation,  to  assess  cost  and 
schedule  performance  and  identify  factors  affecting  that  performance.  We 
also  conducted  semi-structured  interviews  with  program  officials  to 
discuss  the  information  identified  through  reviews  of  program 
documentation  and  obtain  additional  insights  into  factors  that  affected 
program  cost  or  schedule  performance.  Additionally,  we  analyzed  prior 
GAO  reports  to  determine  the  extent  to  which  the  factors  we  identified  as 
affecting  cost  and  schedule  performance  for  selected  ACAT  II  and  III 
programs  were  similar  to  factors  that  we  have  identified  in  prior  work  as 
affecting  performance  of  MDAPs. 

To  address  our  third  objective,  we  reviewed  DOD  acquisition  policy 
related  to  the  reclassification  of  ACAT  II  or  III  programs  to  ACAT  I 
programs  and  analyzed  program  cost  data  provided  by  DOD 
components.  Based  on  the  requirement  in  DOD  acquisition  policy  for 
components  to  notify  the  Defense  Acquisition  Executive  of  ACAT  II  or  III 
programs  within  10  percent  of  the  next  ACAT  level,  we  analyzed  data 
provided  by  DOD  through  our  DCI  to  identify  programs  that  appeared  to 
be  within  10  percent  of  or  have  exceeded  either  the  ACAT  I  RDT&E  or 
procurement  threshold.  We  were  unable  to  identify  an  actual  number  of 
programs  likely  to  become  MDAPs  because  of  reliability  issues  related  to 
identifying  the  population  of  ACAT  II  and  III  programs.  However,  we 
determined  the  initial  data  provided  to  us  by  DOD  that  included  836 
reported  ACAT  II  or  III  programs  were  sufficiently  reliable  to  serve  as  a 
starting  point  to  identify  the  minimum  number  of  programs  likely  to 
become  MDAPs  because  we  were  able  to  confirm  data  with  relevant 
program  offices  for  those  programs  that  appear  to  be  within  10  percent  of 
or  have  exceeded  either  ACAT  I  threshold.  We  excluded  certain 
programs  from  further  review  that  were  known  at  the  time  that  we  initially 
identified  programs  to  have  incorrectly-reported  cost  estimates. 

For  programs  that  appeared  to  meet  our  criteria  for  current  ACAT  II  or  III 
programs  likely  to  become  MDAPs,  we  collected  additional  information 
using  a  structured  set  of  questions  to  determine  whether  the  relevant 
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DOD  component  had  notified  the  Defense  Acquisition  Executive  that  the 
program  was  approaching  or  had  exceeded  the  ACAT  I  threshold  and 
whether  the  program  had  been  or  was  expected  to  be  reclassified  as  an 
ACAT  I  program.  We  also  requested  and  reviewed  supporting 
documentation  when  available,  including  documentation  of  notification  to 
the  Defense  Acquisition  Executive  that  the  program  was  within  10  percent 
of  the  ACAT  I  threshold.  After  we  received  the  information  from  the 
components,  we  identified  additional  programs  that  had  incorrectly 
reported  cost  estimates  or  were  no  longer  current  ACAT  II  or  III  programs 
and  we  removed  these  programs  from  our  analysis  as  appropriate. 

We  conducted  this  performance  audit  from  October  2013  to  March  2015 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Table  7:  Applicability  of  Selected  Acquisition  Program  Milestone  Documentation  Requirements  by  Acquisition  Category 
(ACAT)  Level 


Requirement 

Description 

Major  defense 

acquisition 

program 

Major  automated 

information 

system 

ACAT 

II 

ACAT 

III 

2366a/b  Certification  Memorandum3 

Certifies  that  certain  acquisition  process 
requirements  have  been  fulfilled  prior  to 
milestone  approval 

✓ 

Acquisition  Program 

Baseline 

Summarizes  program  cost,  schedule, 
and  performance  parameters 

Y 

Y 

Y 

Y 

Independent 

Cost  Estimate 

Program  cost  estimate  completed 
outside  of  the  supervision  of  the  entity 
responsible  for  the  acquisition  program 

Y 

Y 

Requirements  Documentation 

Documents  capability  requirements  to 
which  the  program  responds 

Y 

Y 

Y 

Y 

Systems  Engineering  Plan 

Describes  program’s  overall  technical 
approach  and  details  timing  and  criteria 
for  technical  reviews 

✓ 

Y 

Y 

Y 

Technology  Readiness  Assessment 

Assessment  of  the  maturity  of  critical 
technologies  and  related  risks 

Y 

Test  and  Evaluation  Master  Plan 

Primary  planning  and  management  tool 
for  integrated  test  program 

Y 

Y 

Y 

Y 

Source:  GAO  analysis  of  Department  of  Defense  Interim  Instruction  5000.02.  |  GAO-15-188 

a10  U.S.C.  §  2366a  and  b. 


Table  8:  Applicability  of  Selected  Congressional  Reporting  Requirements  by  Acquisition  Category  (ACAT)  Level 


Requirement 

Description 

Major  defense 

acquisition 

program 

Major  automated 

information  ACAT  ACAT 

system  II  III 

Selected  Acquisition  Report/  Major 
Automated  Information  System 
Annual  Report 

Provides  information  on  total  program 
cost,  schedule,  and  performance  to 
Congress 

Y 

Y 

Congressional  Notification  of  Critical 
or  Significant  Cost  Breach 

Provides  notification  of  unit  cost 
breaches  above  a  certain  threshold3 

Y 

Major  Automated  Information 

System  Critical  Change  Report  or 
Significant  Change  Notification 

Provides  notification  of  cost  or  schedule 
changes  above  a  certain  threshold15 

Y 

Source:  GAO  analysis  of  Department  of  Defense  Interim  Instruction  5000.02.  |  GAO-15-188 


a10  U.S.C  §  2433  and  2433a.  Notification  must  be  provided  to  Congress  when  the  program 
acquisition  unit  cost  or  average  procurement  unit  cost  increases  by  at  least  15  percent  over  the 
current  baseline  estimate  or  30  percent  over  the  original  baseline  estimate. 

b10  U.S.C  §  2445c.  Notification  must  be  provided  to  Congress  when  there  is  a  schedule  change  that 
will  cause  a  delay  of  more  than  6  months;  an  increase  in  the  expected  development  cost  or  full  life- 
cycle  cost  for  the  program  by  at  least  15  percent;  or  a  significant,  adverse  change  in  the  expected 
performance  of  the  major  automated  information  system  to  be  acquired. 
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We  conducted  an  analysis  to  determine  whether  data  provided  by 
Department  of  Defense  (DOD)  components  were  sufficiently  reliable  for 
the  purpose  of  determining  the  number,  total  acquisition  cost,  and  cost 
performance  of  DOD’s  current  acquisition  category  (ACAT)  II  and  III 
programs.  For  our  analysis,  we  conducted  electronic  and  manual  testing 
on  data  for  all  programs  reported  by  components  in  response  to  our 
request  for  completion  of  a  data  collection  instrument  (DCI)  and 
compared  cost  data  for  a  non-generalizable  sample  of  programs  to 
source  documents  when  available.  We  also  reviewed  relevant  DOD  and 
component  acquisition  policy,  and  interviewed  knowledgeable  officials. 
We  identified  reliability  issues  with  the  data  for  about  60  percent  of  the 
programs  components  initially  reported  to  us.  As  a  result,  we  determined 
that  the  data  provided  by  DOD  components  were  not  sufficiently  reliable 
to  identify  the  number  of  current  ACAT  II  and  III  programs,  their  estimated 
total  acquisition  cost,  or  the  cost  performance  of  DOD’s  ACAT  II  and  III 
programs. 


Methodology 


To  assess  the  accuracy  and  completeness  of  the  ACAT  II  and  III  program 

data  reported  by  DOD  components,  we  electronically  tested  the  data  for: 

•  values  outside  the  designated  range  of  values  for  ACAT  II  and  III 
programs,  defined  per  DOD  acquisition  policy;1 

•  obvious  calculation  or  data  entry  errors  (for  example,  individual  cost 
elements  do  not  sum  to  total  reported); 

•  missing  data  in  baseline  or  current  cost  estimate  data  elements, 
including  estimates  for  research,  development,  test,  and  evaluation; 
procurement;  military  construction;  and  acquisition  operation  and 
maintenance,  as  well  as  the  total  acquisition  cost  estimate,  and  base 
year;  and 

•  missing  data  in  program  descriptive  data  elements,  such  as  ACAT 
level,  milestone  decision  authority,  or  commodity  type. 


According  to  DOD  policy,  ACAT  II  and  III  programs  are  funded  acquisitions  and  should 
generally  have  cost  estimates  less  than  the  thresholds  for  ACAT  I  programs.  ACAT  I 
thresholds  are  $480  million  in  research,  development,  test,  and  evaluation  or  $2.79  billion 
in  procurement  in  fiscal  year  2014  constant  dollars.  Therefore,  we  expected  that  ACAT  II 
or  III  programs  reported  by  the  components  would  have  cost  estimates  greater  than  zero 
dollars  and  less  than  the  ACAT  I  thresholds. 


Page  39 


GAO-15-188  DOD’s  ACAT  II  and  III  Programs 


Appendix  III:  Observations  on  Reliability  of 
Department  of  Defense  Components’ 
Acquisition  Category  II  and  III  Program  Data 


Additionally,  we  compared  cost  data  for  our  sample  of  170  programs  to 
source  documents  when  available.  Specifically,  for  each  program  in  our 
sample,  we  first  requested  and  reviewed  original  and  current  acquisition 
program  baselines  (APB)  to  determine  whether  or  not  they  reflected  the 
actual  baseline  from  program  start  and  the  current  APB  based  on  the 
approval  date  of  the  APB  and  relevant  schedule  milestones  that  trigger 
the  development  of  an  APB  or  an  APB  revision  in  accordance  with  DOD 
acquisition  policy.2  When  APBs  were  not  provided  or  did  not  appear  to 
reflect  the  actual  baseline  and/or  current  APB,  we  followed  up  with  DOD 
components  to  obtain  the  correct  documents  when  possible.  When  we 
were  able  to  obtain  both  original  and  current  APBs,  we  took  the  following 
steps  to  assess  the  accuracy  of  the  information  reported  in  the  DCI: 

•  Compared  baseline  cost  data  from  the  program’s  original  APB  to  the 
baseline  cost  data  reported  in  the  DCI. 

•  Compared  cost  data  in  the  current  APB  to  cost  data  reported  in  the 
DCI  to  identify  obvious  errors  in  the  cost  data  reported  in  the  DCI, 
such  as  current  cost  data  in  the  DCI  that  was  significantly  less  than 
the  amount  reported  in  the  APB  without  explanation.3 

To  assess  the  consistency  of  ACAT  II  and  III  program  data,  we  manually 
reviewed  the  data  provided  in  response  to  the  DCI  and  subsequent 
requests  to  identify  programs  that  did  not  appear  to  meet  our  criteria  for 
current  ACAT  II  and  III  programs.  For  the  purposes  of  this  report,  we 
defined  a  current  program  as  one  that  has  been  formally  initiated  in  the 
acquisition  process  but  has  not  yet  delivered  90  percent  of  its  planned 
units  or  expended  90  percent  of  its  planned  expenditures.  For  each 
program  that  did  not  appear  to  be  a  current  ACAT  II  or  III  program,  we 
analyzed  whether  the  program  was  pre-program  start,  in  sustainment, 
completed,  or  not  a  separate  ACAT  II  or  III  program  (for  example,  was  a 
subprogram  of  another  ACAT  II  or  III  program  reported  to  us). 


interim  Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition 
System,  (Nov.  25,  2013). 

3We  requested  that  current  cost  information  in  the  DCI  reflect  the  President’s  Fiscal  Year 
2014  budget  request,  which  may  not  align  with  the  time  frame  during  which  a  program’s 
current  APB  was  developed.  Therefore,  we  did  not  expect  that  current  cost  data  reported 
in  the  DCI  would  necessarily  match  cost  data  in  a  program’s  current  APB. 
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We  also  reviewed  relevant  DOD  and  component  policy,  and  interviewed 
agency  officials  responsible  for  acquisition  policy  and  knowledgeable 
about  the  maintenance  of  component-level  data  on  ACAT  II  and  III 
programs.  We  reviewed  these  policies  for  information  such  as  whether 
they  addressed  what  data  should  be  collected  and  maintained  on  ACAT  II 
and  III  programs  or  provided  guidance  to  help  ensure  consistency  of  data 
across  components.  We  met  with  officials  from  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  to 
discuss  relevant  DOD  acquisition  policy,  and  component  officials  from  the 
following  offices  to  discuss  relevant  component  acquisition  policy  and 
efforts  to  collect  ACAT  II  and  III  program  data,  as  well  as  any  efforts  to 
assess  ACAT  II  and  III  program  performance: 

•  Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics 
and  Technology 

•  Office  of  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development  and  Acquisition 

•  Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition 

•  Special  Operations  Research,  Development,  and  Acquisition  Center 

•  Joint  Program  Executive  Office  for  Chemical  and  Biological  Defense 


The  results  of  our  data  reliability  analysis  capture  accuracy, 
completeness,  and  consistency  issues  with  the  data  provided  by  DOD 
components.  Accuracy,  completeness,  and  consistency  are  key 
characteristics  of  reliable  data  and  refer  to  (1 )  the  extent  that  recorded 
data  reflect  the  actual  underlying  information;  (2)  data  elements  for  each 
program  are  populated  appropriately;  and  (3)  the  need  to  obtain  and  use 
data  that  are  clear  and  well  defined  enough  to  yield  similar  results  in 
similar  analyses,  respectively. 

Accuracy  and  Completeness  We  identified  numerous  types  of  accuracy  and  completeness  issues  with 

the  data  provided  by  DOD  components,  including  cost  estimate  values 
outside  of  the  ACAT  II  and  III  program  range,  basic  math  errors,  and 
missing  data.  For  example,  333  out  of  836  ACAT  II  and  III  programs 
reported  by  the  components  were  missing  a  baseline  or  current  cost 
estimate  element  or  descriptive  program  data.  Table  9  provides  detail  on 
accuracy  and  completeness  issues  by  component. 


Results  of  Analysis 
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Table  9:  Accuracy  and  Completeness  Issues  Identified  by  GAO  in  Department  of  Defense  Components’  Acquisition  Category 
(ACAT)  II  and  III  Program  Data 


U.S.  Special 


Issues  identified  by  GAO 

Army 

(n=307) 

Navy 

(n=110) 

Air 

Force 

(n=338) 

Operations 

Command 

(n=65) 

Chemical  and  Biological 
Defense  Program 
(n=16) 

Values  outside  ACAT  II  and  III  program  range 
(63  programs)3 

10 

11 

34 

8 

0 

Basic  math  errors  (54  programs)3 

6 

14 

21 

8 

5 

Missing  cost  estimate  data  (315  programs) 

177 

19 

110 

9 

0 

Missing  descriptive  program  data  (43  programs) 

14 

2 

27 

0 

0 

Source:  GAO  analysis  of  Department  of  Defense  component  data.  |  GAO-15-188 

Notes:  We  identified  these  issues  through  electronic  testing  of  the  data.  We  conducted  separate  tests 
for  each  issue,  so  a  single  program  may  be  reported  in  more  than  one  category. 

a  According  to  DOD  policy,  ACAT  II  and  III  programs  are  funded  acquisitions  and  should  be  less  than 
the  thresholds  for  ACAT  I  programs.  ACAT  I  thresholds  are  $480  million  in  research,  development, 
test,  and  evaluation  or  $2.79  billion  in  procurement  in  fiscal  year  2014  constant  dollars.  Therefore,  we 
expected  that  all  ACAT  II  and  III  programs  would  have  cost  estimates  greater  than  zero  dollars  and 
less  than  ACAT  I  thresholds. 

bBasic  math  errors  include  programs  with  current  or  baseline  cost  estimate  elements  that  when  added 
up  and  rounded  to  the  nearest  million  do  not  match  the  reported  total  cost  estimate  rounded  to  the 
nearest  million.  This  category  excluded  programs  missing  baseline  or  current  cost  estimates. 


We  identified  additional  issues  with  the  accuracy  of  the  ACAT  II  and  III 
program  cost  information  when  we  compared  reported  cost  estimates  to 
available  source  documents  for  a  non-generalizable  sample  of  ACAT  II 
and  III  programs.  Specifically,  of  the  81  programs  in  our  sample  that 
reported  complete  cost  estimates  and  provided  source  documents,  50 
reported  incorrect  cost  data.  For  example,  for  37  of  these  50  programs, 
we  determined  that  baseline  cost  data  was  inaccurate  because  either  the 
baseline  cost  estimate  or  the  base  year  reported  for  this  estimate  did  not 
match  the  source  documents.  Details  of  the  accuracy  and  completeness 
issues  we  identified  when  assessing  the  cost  data  reported  for  our 
sample  are  provided  by  component  in  table  10. 
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Table  10:  Issues  Found  When  Comparing  Reported  Cost  Data  to  Source  Documents  for  Non-generalizable  Random  Sample  of 


Acquisition  Category  (ACAT)  II  and  III  Programs 

Cost  Data  Issues 

Total  for  5 
Components 

Army 

Navy 

Air 

Force 

U.S.  Special 
Operations 
Command 

Chemical  and 
Biological  Defense 
Program 

Missing  one  or  more  acquisition  program  baseline 
document3 

75 

10 

9 

38 

13 

5 

Missing  baseline  or  current  cost  estimate  element 

14 

14 

0 

0 

0 

0 

Partial  cost  estimate 

2 

0 

1 

0 

1 

0 

Data  error  in  cost  estimate  element 

48 

10 

13 

6 

10 

9 

No  reliability  issues  identified 

31 

11 

14 

4 

2 

0 

Total  programs  in  sample 

170 

45 

37 

48 

26 

14 

Source:  GAO  analysis  of  Department  of  Defense  component-provided  acquisition  program  baselines  and  cost  estimates.  |  GAO-15-188 

aWe  did  not  analyze  data  reliability  issues  for  programs  that  lacked  one  or  more  acquisition  program 
baseline  documents  since  these  documents  were  considered  to  be  the  source  document  for  the 
baseline  cost  data  provided. 


Consistency  We  identified  226  of  the  836  programs  reported  by  DOD  components  that 

did  not  meet  our  criteria  for  current  ACAT  II  or  III  programs  because,  for 
example,  they  were  not  current  or  they  were  not  stand-alone  acquisition 
programs.  Additionally,  because  information  on  program  phase  was  not 
available  for  all  programs  reported  by  DOD  components,  the  number  of 
programs  we  identified  as  not  a  current  ACAT  II  or  III  program  reflects  a 
minimum  number  of  such  programs.  Table  11  provides  details  on 
consistency  issues  we  identified  by  component. 


Table  11:  Consistency  Issues  Identified  in  Department  of  Defense  Components’  Acquisition  Category  (ACAT)  II  and  III 

Program  Data 

Army 

(n=307) 

Navy 
(n=1 10) 

Air 

Force 

(n=338) 

U.S.  Special 
Operations  Command 
(n=65) 

Chemical  and  Biological 
Defense  Program 
(n=16) 

Not  a  current  program  (pre-program  start,  in 
sustainment,  or  completed)  (199  programs) 

120 

8 

68 

3 

0 

Not  a  separate  program  or  other  reason 
(27  programs)3 

4 

3 

6 

14 

0 

Sum  of  non-current/non-ACAT  II  and  III 
programs  (226  programs) 

124 

11 

74 

17 

0 

Source:  GAO  analysis  of  Department  of  Defense  component  data.  |  GAO-15-188 

“Programs  categorized  as  “other  reason”  include  programs  led  by  a  component  other  than  the 
reporting  component  or  programs  for  which  the  reporting  component  provided  sufficient  information  to 
determine  the  program  was  not  a  current  ACAT  II  or  III  program,  but  not  enough  information  to 
determine  the  specific  program  phase  (e.g.  pre-program  start,  in  sustainment,  or  completed). 
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Figure  4:  Description  of  Selected  Army  Acquisition  Category  (ACAT)  II  and  III  Programs 


ACAT  LEVEL  II:  AN/TPQ-53  Counterfire  Target  Acquisition  Radar  System 

Program  Executive  Office:  Missiles  and  Space 

Estimated  Total  Acquisition  Cost  as  of  March  2012:  $2.7  billion 

The  AN/TPQ-53  Counterfire  Target  Acquisition  Radar  System  provides  warfighters  with  continuous  and  responsive 
counter-battery  target  acquisition  capabilities  for  all  types  of  military  operations  and  replaces  the  legacy  AN/TPQ-36 
and  AN/TPQ-37  target  acquisition  counterfire  radar  systems.  It  detects  in-flight  projectiles  and  determines  and 
communicates  firing  point  locations  for  mortars,  artillery,  and  rockets  with  a  high  degree  of  accuracy  and  low  false 
alarm  rates.  The  AN/TPQ-53  program  was  initiated  in  2006  and  was  approved  to  start  production  to  meet  an  urgent 
warfighter  need  in  July  2008.  The  Army  expects  to  field  a  total  of  178  systems.3 


ACAT  LEVEL  III:  M855A1  5.56  mm  Enhanced  Performance  Round 

Program  Executive  Office:  Ammunition 

Estimated  Total  Acquisition  Cost  as  of  February  2011 :  $2.4  billion 

The  M855A1  5.56mm  Enhanced  Performance  Round  is  a  lead-free  version  of  the  M855  cartridge  fired  from  the  M4 
carbine,  M16  rifle,  and  M249  Squad  Automatic  Weapon  machine  gun.  According  to  Army  officials,  the  design  of  the 
M855A1  has  resulted  in  several  performance  enhancements,  including  better  hard  target  penetration,  more 
consistent  performance  against  soft  targets,  and  better  outcomes  for  these  effects  at  significantly  increased 
distances  as  compared  to  the  original  M855  cartridge.  The  M855A1  was  approved  for  fielding  as  an  engineering 
change  proposal  to  the  original  M855  cartridge  in  June  2010.  As  of  December  2014,  over  one  billion  rounds  had 
been  produced. 


ACAT  LEVEL  III:  Synthetic  Environment  Core 

Program  Executive  Office:  Simulation,  Training,  and  Instrumentation 
Estimated  Total  Acquisition  Cost  as  of  October  2011:  $161.2  million 

The  Synthetic  Environment  Core  program  includes  development  efforts  for  new  components  and  the  integration  of 
existing  software  and  hardware  products  to  create  the  Army's  Common  Virtual  Environment.  These  components  will 
be  available  for  integration  into  host  virtual  training  aids,  devices,  simulators,  and  simulations  that  will  undergo 
development  or  upgrades  based  on  emerging  requirements.  The  program  also  includes  the  development  of  a  rapid 
terrain  generation  process  and  the  establishment  of  associated  database  facilities.  Synthetic  Environment  Core  was 
initiated  as  an  ACAT  III  program  at  development  start  in  March  2005,  but  it  is  currently  being  re-designated  an  ACAT 
II  program  due  to  cost  increases. 


Source:  GAO  analysis  of  Army  program  documents  (data);  Lockheed  Martin  Mission  Systems  &  Training  (top  photo)  and  U.S.  Army  (middle  and  bottom  photos).  |  GAO-15-188 

Notes:  All  dollar  figures  are  provided  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 
Estimated  total  acquisition  cost  was  taken  from  each  program’s  most-recently  approved  acquisition 
program  baseline.  A  program’s  current  total  acquisition  cost  estimate  may  differ  from  the  amount 
reported  here  if  the  estimate  has  changed  since  the  program’s  acquisition  program  baseline  was 
approved. 

aThe  initial  operational  capability  date  for  the  AN/TPQ-53  Counterfire  Target  Acquisition  Radar 
System  is  considered  by  the  Army  to  be  sensitive  and  is  therefore  not  included  in  this  report. 
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Figure  5:  Description  of  Selected  Navy  Acquisition  Category  (ACAT)  II  and  III  Programs 


ACAT  LEVEL  II:  AN/SPY-1  D(V)  Radar  Upgrade 

Program  Executive  Office:  Integrated  Warfare  Systems 
Estimated  Total  Acquisition  Cost  as  of  February  2012:  $304.8  million 

The  AN/SPY-1  D(V)  Radar  Upgrade  program  improves  the  performance  of  the  AN/SPY-1  D(V)  radar,  a  high-powered 
multi-functional  phased-array  radar  capable  of  performing  search,  track,  and  missile  guidance  functions 
simultaneously  in  both  littoral  and  deepwater  environments.  The  AN/SPY-1  D(V)  radar  upgrade  program  was  initiated 
at  development  start  in  September  1993.  The  Navy  fielded  its  initial  capability  in  2004  and  expects  to  field  a  total  of 
35  systems.  The  program  is  nearly  complete  and  the  Navy  expects  the  Air  and  Missile  Defense  Radar  program  to 
provide  radar  resources  for  DDG-51  class  ships  going  forward. 


ACAT  LEVEL  III:  Nulka  Shipboard  Improvement 

Program  Executive  Office:  Integrated  Warfare  Systems 
Estimated  Total  Acquisition  Cost  as  of  April  2009:  $1.1  billion 

The  Nulka  Shipboard  Improvement  program  is  a  rocket-launched  active  electronic  warfare  countermeasure  system 
designed  to  defend  surface  combatants  against  anti-ship  cruise  missiles.  It  processes  multiple  threats  simultaneously 
from  either  data  entered  manually  by  the  operator  or  inputted  automatically  by  the  ship’s  Combat  System  and 
Electronic  Support  Measures  equipment.  The  program  was  initiated  at  development  start  in  March  1987  and 
expected  to  deliver  a  total  of  163  launch  systems  and  1 ,288  decoy  cartridges  as  of  the  program’s  most  recent 
acquisition  program  baseline  in  April  2009. 


Marine  Corps  Family  of  Light  Armored  Vehicles  -  Anti-Tank  Modernization  and  Command  and  Control  Upgrade  Programs 

Program  Executive  Office:  Marine  Corps  Systems  Command 

The  Family  of  Light  Armored  Vehicles  (LAV)  is  an  8x8  wheeled  family  of  vehicles  comprised  of  a  base  variant  that  was  initially  fielded  in  1983  as  the  LAV-25  and  six  mission 
role  variants.  The  vehicles  are  intended  to  provide  combined  arms  reconnaissance,  security  missions,  and  mobile  electronic  support. 


ACAT  LEVEL  III:  LAV 
Anti-Tank  Modernization 

Estimated  Total  Acquisition 
Cost  as  of  June  2013: 
$339.0  million 


The  LAV  Anti-Tank 
Modernization  program  is 
expected  to  enable  the 
Marine  Corps  to  acquire 
and  destroy  main  battle 
tanks  when  Light  Armored 
Reconnaissance  units  face  an  enemy  armored  formation  threat.  The  anti-tank 
variant  is  expected  to  be  capable  of  firing  multiple  shots  and  acquiring  targets  while 
on  the  move  with  an  improved  thermal  sight  and  advanced  fire  control  system.  This 
program  was  initiated  at  development  start  in  May  2011  and  plans  to  deliver  the 
initial  capability  in  fiscal  year  2018  and  procure  a  total  of  118  systems. 


ACAT  LEVEL  IIP:  LAV 
Command  and  Control 
Upgrade 

Estimated  Total  Acquisition 
Cost  as  of  November  2009: 
$114.5  million 


The  LAV  Command  and 
Control  Upgrade  program  is 
a  suite  of  vehicle 
communication  upgrades 
including  the  ability  to 
receive  and  transmit  data  between  vehicles  and  share  information  among  computer 
workstations.  It  also  is  designed  to  incorporate  more  reliable  and  mobile  long  range 
high-frequency  and  satellite  communication  capabilities.  The  program  was  initiated 
at  development  start  in  March  2005,  delivered  its  initial  capability  in  September 
2011,  and  as  of  August  2012,  had  delivered  55  of  69  total  planned  systems. 


Source:  GAO  analysis  of  Navy  program  documents  (data);  Senior  Chief  Donnie  Ryan,  U.S.  Navy  (top  photo),  Navy  Program  Executive  Office  Integrated  Warfare  Systems  (middle  photo),  and 
U.S.  Marine  Corps  (bottom  photos).  |  GAO-15-188 


Notes:  All  dollar  figures  are  provided  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 
Estimated  total  acquisition  cost  was  taken  from  each  program’s  most-recently  approved  acquisition 
program  baseline.  A  program’s  current  total  acquisition  cost  estimate  may  differ  from  the  amount 
reported  here  if  the  estimate  has  changed  since  the  program’s  acquisition  program  baseline  was 
approved. 

aThe  LAV  Command  and  Control  Upgrade  program  is  considered  to  be  an  ACAT  IV  program  by  the 
Marine  Corps.  For  the  purposes  of  this  report,  we  considered  the  Navy’s  ACAT  IV  programs  to  be 
ACAT  III  programs  since  DOD  only  recognizes  ACAT  I,  II,  and  III  programs. 
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Figure  6:  Description  of  Selected  Air  Force  Acquisition  Category  (ACAT)  II  and  III  Programs 


ACAT  LEVEL  II:  F-15E  Radar  Modernization  Program 

Program  Executive  Office:  Fighters  and  Bombers 
Estimated  Total  Acquisition  Cost  as  of  May  2014:  $2.9  billion 

The  F-1 5E  Radar  Modernization  Program  replaces  the  F-1 5E  legacy  APG-70  mechanically  scanned  radar  with  an 
active  electronically  scanned  array  system  designed  to  retain  functionality  of  the  legacy  radar  system  while  providing 
expanded  mission  employment  capabilities  including  improved  operational  capabilities  for  long-range  detection, 
tracking,  and  mapping  in  air-to-air  and  air-to-ground  environments.  The  program  was  initiated  at  development  start  in 
September  2008  and  the  Air  Force  plans  to  procure  222  systems  and  deliver  the  initial  capability  to  the  warfighter  in 
August  201 5. 


ACAT  LEVEL  III:  Air  Force  Subscale  Aerial  Target 

Program  Executive  Office:  Weapons 

Estimated  Total  Acquisition  Cost  as  of  February  2010:  $968.9  million 

The  Air  Force  Subscale  Aerial  Target  program  is  a  turbojet  powered,  subscale  aerial  target  system  consisting  of  an 
aircraft,  payload,  and  support  equipment.  It  uses  rocket-assisted  takeoffs  from  the  ground  to  conduct  various 
scenarios  and  employs  countermeasure  techniques  to  simulate  adversarial  threats  and  tactics  for  weapons 
evaluation  and  operator  familiarization.  The  program  was  initiated  at  production  start  in  July  2004  and  the  Air  Force 
expects  to  field  800  systems.  The  initial  capability  was  delivered  to  the  warfighter  in  December  2007. 


ACAT  LEVEL  III:  Haystack  Ultra-Wideband  Satellite  Imaging  Radar 

Program  Executive  Office:  Space 

Estimated  Total  Acquisition  Cost  as  of  April  2013:  $107.5  million 

The  Haystack  Ultra-Wideband  Satellite  Imaging  Radar  program  was  an  upgrade  to  the  existing  X-band  Haystack 
Imaging  Radar,  operated  by  Massachusetts  Institute  of  Technology  Lincoln  Laboratory  for  the  Air  Force.  The  upgrade 
adds  W-band  capability  and  enhances  imaging  resolution  to  characterize  smaller  objects  in  low  earth  orbit  and  add 
deep-space  tracking  capability.  The  Air  Force  initiated  the  program  as  a  low-cost  modification  at  development  start  in 
July  2004.  The  upgrade  was  completed  and  accepted  by  the  Air  Force  in  February  2014. 


Source:  GAO  analysis  of  Air  Force  program  documents  (data);  Boeing  {top  photo)  and  U.S.  Air  Force  (middle  and  bottom  photos).  |  GAO-1 5-1 88 

Notes:  All  dollar  figures  are  provided  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 
Estimated  total  acquisition  cost  was  taken  from  each  program’s  most-recently  approved  acquisition 
program  baseline.  A  program’s  current  total  acquisition  cost  estimate  may  differ  from  the  amount 
reported  here  if  the  estimate  has  changed  since  the  program’s  acquisition  program  baseline  was 
approved. 
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Figure  7:  Description  of  Selected  U.S.  Special  Operations  Command  (SOCOM)  Acquisition  Category  (ACAT)  II  and  III 
Programs 


ACAT  LEVEL  II:  AC-130J 

Program  Executive  Office:  Fixed  Wing 

Estimated  Total  Acquisition  Cost  as  of  October  2013:  $2.1  billion 

The  AC-130J  program  is  designed  to  be  a  modular,  scalable  weapon  system  that  maximizes  the  full  spectrum  of 
available  weapons  through  full  use  of  the  AC-1 30J  aircraft  with  the  primary  missions  of  close  air  support,  air 
interdiction,  and  other  associated  collateral  missions.  SOCOM  expects  the  AC-1 30J  and  its  associated  precision 
strike  package  to  provide  combatant  commands  with  increased  operational  availability  and  fire  support  capabilities. 
The  program  was  initiated  at  development  start  in  March  2012,  and  SOCOM  expects  to  field  37  systems  and  deliver 
the  initial  capability  to  the  warfighter  by  March  2017. 


ACAT  LEVEL  III:  Nonstandard  Aviation 

Program  Executive  Office:  Fixed  Wing 

Estimated  Total  Acquisition  Cost  as  of  July  2009:  $480.0  million 

The  Nonstandard  Aviation  program  objective  is  to  acquire,  modify,  field,  and  sustain  commercial  aircraft  for  SOCOM 
and  the  Air  Force  Special  Operations  Command.  Program  efforts  also  include  procuring  initial  spares  and  training 
aircrews  and  maintenance  personnel  on  the  nonstandard  aviation  mission  to  support  Special  Operations  Forces 
counter-terrorism  operations.  The  Nonstandard  Aviation  program  was  initiated  at  production  start  in  July  2009. 
SOCOM  expects  to  deliver  a  total  of  23  aircraft  by  September  2018. 


ACAT  LEVEL  III:  Special  Operations  Forces  Deployable  Node  Family  of  Systems 

(Heavy,  Medium,  and  Light) 

Program  Executive  Office:  Command,  Control,  Communications,  and  Computers 
Estimated  Total  Acquisition  Cost  as  of  September  201 3:  $1 56.7  million" 

The  Special  Operations  Forces  Deployable  Node  Family  of  Systems  is  a  family  of  deployable  land 
or  sea  satellite  communication  systems  providing  deployed  special  operations  forces  user 
elements  and  teams  with  access  to  the  Special  Operations  Forces  Information  Environment  for 
high-capacity  voice,  data,  video-teleconference,  and  video  at  all  levels  of  classification.  These 
satellite  communication  systems  with  associated  switching  equipment  and  terminals  have  super 
high  frequencies  and  multiple  sizes  to  support  large,  medium,  and  small  special  operations  forces 
operational  elements  using  the  heavy,  medium,  and  light  variants,  respectively.  As  of  April  2014, 
SOCOM  had  delivered  58  Special  Operations  Forces  Deployable  Node-Heavy  systems,  208 
Medium  systems,  and  860  Light  systems. 


Source:  GAO  analysis  of  SOCOM  program  documents  (data);  SOCOM  (photos).  |  GAO-15-188 

Notes:  All  dollar  figures  are  provided  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 
Estimated  total  acquisition  cost  was  taken  from  each  program’s  most-recently  approved  acquisition 
program  baseline  unless  otherwise  noted.  A  program's  current  total  acquisition  cost  estimate  may 
differ  from  the  amount  reported  here  if  the  estimate  has  changed  since  the  program’s  acquisition 
program  baseline  was  approved. 

aThe  Special  Operations  Forces  Deployable  Node  Family  of  Systems  cost  estimate  was  taken  from 
the  program’s  September  2013  Life  Cycle  Cost  Estimate.  The  figure  reflects  fiscal  year  2012  through 
201 9  only  and  excludes  $654.3  million  of  Capital  Equipment  Replacement  Program  funding  included 
in  the  program’s  budget. 
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Figure  8:  Description  of  Selected  Chemical  and  Biological  Defense  Program  (CBDP)  Acquisition  Category  (ACAT)  II  and  III 
Programs 


ACAT  LEVEL  II:  Joint  Biological  Point  Detection  System 

Program  Executive  Office:  Chemical  and  Biological  Defense 
Estimated  Total  Acquisition  Cost  as  of  September  2011:  $777.4  million 

The  Joint  Biological  Point  Detection  System  program  increases  detection  sensitivity  and  the  number  of  agents  that 
are  identified,  decreases  detection  and  identification  time,  and  provides  automated  detection  and  identification.  It 
consists  of  complementary  detector,  collector,  and  identification  technologies  to  rapidly  and  automatically  detect  and 
identify  biological  threat  agents.  The  program  was  initiated  at  development  start  in  December  1996.  CBDP  planned 
to  field  1 ,063  systems  and  delivered  initial  capability  to  the  warfighter  in  August  2003. 


ACAT  LEVEL  III:  Dismounted  Reconnaissance  Sets,  Kits,  and  Outfits 

Program  Executive  Office:  Chemical  and  Biological  Defense 
Estimated  Total  Acquisition  Cost  as  of  April  2013:  $1 .0  billion 

The  Dismounted  Reconnaissance  Sets,  Kits,  and  Outfits  program  is  expected  to  provide  commanders  more  detailed 
information  reports  for  use  against  chemical,  biological,  radiological,  and  nuclear  hazards  and  toxic  industrial 
chemicals  on  the  integrated  battlefield.  The  program  was  initiated  at  development  start  in  March  2011.  CBDP  plans 
to  field  601  systems  and  deliver  initial  capabilities  to  the  warfighter  by  service,  starting  with  the  Navy  in  June  2014; 
the  Air  Force  and  Army  in  June  201 5;  and  the  Marine  Corps  in  March  2016. 


Joint  Service  Aircrew  Mask  (JSAM)  Family  of  Systems 

Program  Executive  Office:  Chemical  and  Biological  Defense 

The  Joint  Service  Aircrew  Mask  variants  are  expected  to  provide  individual  head,  eye,  respiratory,  and  chemical-biological  protection  across  numerous  fixed  and  rotary  wing 
platforms.  The  Joint  Service  Aircrew  Mask  Program  was  initiated  at  development  start  in  November  2002  as  a  single  program,  but  the  program  has  since  been  restructured 
multiple  times  to  include  programs  focused  on  specific  variants. 


ACAT  LEVEL  III:  Joint 
Service  Aircrew  Mask  - 
Rotary  Wing 

Estimated  Total  Acquisition 
Cost  as  of  October  2011: 
$202.6  million 


Joint  Service  Aircrew  Mask 


-  Rotary  Wing  program  is 
intended  for  general 
purpose  rotary  wing  aircraft 
and  is  expected  to  address  aircrew  physiological  performance  degradation, 
restrictions  to  field-of-view,  and  provide  don  in-flight  capability.  The  program  was 
created  in  2008  during  a  restructuring  of  the  JSAM  program.  CBDP  plans  to  field 
20,446  systems  and  provide  the  initial  capability  to  the  warfighter  in  December  2015. 


ACAT  LEVEL  III:  Joint 
Service  Aircrew  Mask  - 
Fixed  Wing 

Estimated  Total  Acquisition 
Cost  as  of  November  201 0: 
$56.8  million 

The  Joint  Service  Aircrew 
Mask  -  Fixed  Wing  program 
was  created  in  2008  during 
a  restructuring  of  the  JSAM 
program,  but  was  terminated  in  March  2014  due  to  concerns  about  the  feasibility  of 
meeting  requirements.  Selected  program  requirements  will  be  addressed  through 
two  new  programs:  Joint  Service  Aircrew  Mask  -  Strategic  Aircraft  and  Joint  Service 
Aircrew  Mask  -  Tactical  Aircraft. 


Source:  GAO  analysis  of  CBDP  program  documents  (data);  Joint  Program  Executive  Office  for  Chemical  and  Biological  Defense  (photos).  |  GAO-15-188 

Notes:  All  dollar  figures  are  provided  in  then-year  dollars  and  have  not  been  adjusted  for  inflation. 
Estimated  total  acquisition  cost  was  taken  from  each  program’s  most-recently  approved  acquisition 
program  baseline.  A  program’s  current  total  acquisition  cost  estimate  may  differ  from  the  amount 
reported  here  if  the  estimate  has  changed  since  the  program’s  acquisition  program  baseline  was 
approved. 
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ASSISTANT  SECRETARY  OF  DEFENSE 
3600  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3600 


FE3  5  2015 

Mr.  Michael  J.  Sullivan 

Director.  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report,  GAO-15- 
1 88.  “DEFENSE  ACQUISITIONS:  Better  Approach  Needed  to  Account  for  Number.  Cost,  and 
Performance  of  Non-Major  Programs,"  dated  January  7,  2015  (GAO  Code  121 170).  Comments 
on  the  report  recommendations  are  enclosed. 


Sincerely, 


Enclosure: 
As  stated 


ACQUISITION 
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GAO  DRAFT  REPORT  DATED  JANUARY  7, 2015 
GAO-15-188  (GAO  CODE  121170) 

“DEFENSE  ACQUISITIONS:  BETTER  APPROACH  NEEDED  TO  ACCOUNT  FOR 
NUMBER,  COST,  AND  PERFORMANCE  OF  NON-MAJOR  PROGRAMS” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION:  To  improve  DoD’s  ability  to  collect  and  maintain  reliable  data  on  its 
acquisitions,  the  GAO  recommend  that  the  Secretary  of  Defense  direct  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics,  in  consultation  with  DoD  components,  to 
take  the  following  actions: 

•  establish  guidelines  on  what  constitutes  a  "current''  ACAT  II  or  III  program  for  reporting 
purposes;  the  types  of  programs,  if  any,  that  do  not  require  ACAT  designations;  and 
whether  the  rules  for  identifying  current  MDAPs  would  be  appropriate  for  ACAT  II  and 
III  programs;  and 

•  determine  what  metrics  should  be  used  and  what  data  should  be  collected  on  ACAT  II 
and  III  programs  to  measure  cost  and  schedule  performance;  and  whether  the  use  of 
DAMIR  and  the  MDAP  selected  acquisition  report  format  may  be  appropriate  for 
collecting  data  on  ACAT  II  and  III  programs. 

DoD  RESPONSE;  DoD  partially  concurs  with  the  above  recommendation.  Within  one  year  of 
publication  of  the  final  GAO  report,  the  USD(AT&L),  on  behalf  of  the  Secretary  of  Defense,  in 
consultation  with  the  Service  and  Component  Acquisition  Executives,  will  review  the  existing 
policy  direction  for  ACAT  II  and  III  programs  to  determine  whether  it  needs  to  be  altered  or 
supplemented  to  facilitate  data  collection  and/or  reporting. 

RECOMMENDATION:  The  GAO  recommends  that  the  Secretary  of  Defense  direct  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy  and  the  Commander  of  SOCOM  to  take  the 
following  actions: 

•  assess  the  reliability  of  data  collected  on  ACAT  II  and  III  programs  and  work  with  PEOs 
to  develop  a  strategy  to  improve  procedures  for  the  entry  and  maintenance  of  data;  and 

•  develop  implementation  plans  to  coordinate  and  execute  component  initiatives  to 
improve  data  on  ACAT  II  and  III  programs. 

DoD  RESPONSE:  DoD  partially  concurs  with  the  above  recommendation.  Within  90  to  120 
days  of  publication  of  the  final  GAO  report,  the  USD(AT&L),  on  behalf  ot  the  Secretary  of 
Defense,  will  direct  the  DoD  Components  to  evaluate  the  data  collected  on  ACAT  II  and  III 
programs  and  report  back  to  him  on  their  assessment  of  the  reliability  of  that  data  and  the  status 
of  their  plans  to  improve  the  availability  and  quality  of  the  data. 
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Appendix  V:  Comments  from  the  Department 
of  Defense 


RECOMMENDATION:  To  help  ensure  compliance  with  relevant  provisions  of  DoD 
acquisition  policy  with  the  purpose  of  improving  DoD's  ability  to  provide  oversight  for  ACAT  II 
and  III  programs,  including  those  programs  that  may  become  MDAPs.  the  GAO  recommend  that 
the  Secretary  of  Defense: 

•  direct  the  Secretary  of  the  Air  Force  and  Commander  of  SOCOM  to  establish  a 
mechanism  to  ensure  compliance  with  APB  requirements  in  DoD  policy. 

•  direct  the  Secretaries  of  the  Air  Force,  Army,  and  Navy  to  improve  component 
procedures  for  notifying  the  Defense  Acquisition  Executive  of  programs  whose  cost 
estimate  is  within  10  percent  of  ACAT  1  cost  thresholds. 

DoD  RESPONSE:  DoD  partially  concurs  with  the  above  recommendation.  Within  90  to  120 
days  of  publication  of  the  final  GAO  report,  the  USD(AT&L),  on  behalf  of  the  Secretary  of 
Defense,  will  issue  guidance  to  all  of  the  DoD  Components  reiterating  the  APB  requirements  for 
ACAT  II  and  III  programs  and  directing  that  the  DAE  be  notified  when  an  increase  or  estimated 
increase  in  program  cost  is  within  10  percent  of  the  ACAT  I  cost  thresholds. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO’s  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no 
cost  is  through  GAO’s  website  (http://www.gao.gov).  Each  weekday 
afternoon,  GAO  posts  on  its  website  newly  released  reports,  testimony, 
and  correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted 
products,  go  to  http://www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  website, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Website:  http://www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov,  (202)  512- 
4400,  U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room 
7125,  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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